Storage of backup data using a time-series data lake

ABSTRACT

Techniques are disclosed relating to the storage of backup data using a time-series data lake. For example, in various embodiments, the disclosed techniques include providing a cloud-based data lake service that maintains data for a plurality of organizations and where, for a first organization, the cloud-based data lake service maintains a time-series data lake that stores a time-series representation of data associated with the first organization. In various embodiments, the data lake service may receive backup data from a plurality of data sources associated with the first organization, generate metadata associated with the backup data, and store the backup data, along with the corresponding metadata, in the time-series data lake.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional Application No.62/982,970 entitled “Time-Series Data Lake,” filed on Feb. 28, 2020,which is hereby incorporated by reference as if entirely set forthherein. This application is also related to the following U.S.Applications filed on Feb. 26, 2021: U.S. application Ser. No.17/187,300 (Attorney Docket Number 7251-00202), U.S. application Ser.No. 17/187,353 (Attorney Docket Number 7251-00203), U.S. applicationSer. No. 17/187,359 (Attorney Docket Number 7251-00204), and U.S.application Ser. No. 17/187,365 (Attorney Docket Number 7251-00205).Each of the above-referenced applications is hereby incorporated byreference as if entirely set forth herein.

BACKGROUND Technical Field

The present disclosure relates generally to data storage systems, andmore particularly to the storage of backup data using a time-series datalake.

Description of the Related Art

Organizations often utilize large amounts of data to support businessapplications and services. In some instances, much of an organization'sdata may reside in various different data stores, such as structured orsemi-structured databases, collections of electronic files, data frommessaging systems, etc. Further, in many cases, an organization willmaintain its data in “data silos” in which some of the organization'sdata is isolated from, and thus not accessible to, other parts of theorganization's system. The data stores utilized by an organization aretypically used for online transaction processing, for example to storeand retrieve data as-needed to support the services the organizationprovides, but are typically not effective for online analyticalprocessing (“OLAP”) operations. For example, it may take an extendedtime period (e.g., hours or days) to execute an OLAP query using thedata stores typically utilized by an organization (e.g., relationaldatabases). Accordingly, when an organization needs to perform dataanalytics across such data silos, the organization typically utilizes aseparate data warehousing system. To utilize a data warehouse,organizations traditionally extract data from their various data stores,transform the format and structure of the data as needed to fit thetarget data warehouse, and load that transformed data into the datawarehouse. This process is both time-consuming and computationallyexpensive. Further, since this process results in a point-in-time viewof the organization's data, this “ETL” process must be repeated eachtime the organization wishes to obtain a different view of its data,resulting in significant expense in time and computational resources.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating an example data protectionservice that includes a data lake service, according to someembodiments.

FIG. 2 is a block diagram illustrating an example data protectionservice, according to some embodiments.

FIG. 3A is a block diagram illustrating an example embodiment in which adata protection service receives backup data in the form of a physicalbackup, according to some embodiments. FIG. 3B is a block diagramillustrating an example embodiment in which a data protection servicereceives backup data in the form of a logical backup, according to someembodiments.

FIGS. 4A-4B are flow diagrams illustrating example methods for storingbackup data, along with one or more items of metadata, in a time-seriesdata lake, according to some embodiments.

FIG. 5 is a flow diagram illustrating an example method for retrievingdata from a time-series data lake, according to some embodiments.

FIG. 6 is a block diagram illustrating an example data protectionservice that is operable to provide selected data views, from atime-series data lake, to a data warehousing system, according to someembodiments.

FIG. 7 is a flow diagram illustrating an example method for providingselected data views, from a time-series data lake, to a data warehousingsystem, according to some embodiments.

FIG. 8 is a block diagram illustrating an example data protectionservice that includes a data preservation service, according to someembodiments.

FIG. 9 is a flow diagram that illustrates an example method forpreserving data from a time-series data lake, according to someembodiments.

FIG. 10 is a block diagram illustrating an example data protectionservice that includes a data modification service, according to someembodiments.

FIG. 11 is a flow diagram illustrating an example method for modifyingdata in a time-series data lake, according to some embodiments.

FIG. 12 is a block diagram illustrating an example computer system,according to some embodiments.

DETAILED DESCRIPTION

Organizations often utilize large amounts of data, for example tosupport business applications and services that the organizations runfrom their on-site data centers or using public cloud service providers.In some instances, much of the data for an enterprise organization mayreside in various data stores. As used herein, the term “data store”refers to an organized collection of data. In various embodiments, adata store may be a database, such as a structured or semi-structureddatabase, a collection of electronic files, data from messaging systemssuch as e-mail systems or chat-based systems, data warehouses, socialmedia feeds, file servers, object-based data storage systems, etc.Within a given organization's system, there could be 10s to 1000s ofdata stores, potentially utilizing multiple data storage formats (e.g.,Oracle™ databases, Amazon™ Relational Data Base Service (RDS) databases,Amazon™ DynamoDB databases, Amazon™ Neptune databases, Amazon™ Auroradatabases, Microsoft™ Azure SQL databases, MongoDB™ databases, IBM™ Db2databases, Hadoop™ Distributed File Systems, Microsoft™ Exchangee-mails, Slack™ messages, Microsoft™ 365 Exchange, Microsoft™ 365SharePoint, Microsoft™ 365 OneDrive, Twitter™ feeds, Facebook™ posts,Google™ Suite Gmail, Google™ Suite Drive, Dropbox™ files, Box™ files,etc.).

The data stores utilized by an organization may be optimized for onlinetransaction processing (“OLTP”), storing and retrieving data as neededto support the business application services the organization provides.An organization will typically perform data backup operations topreserve backup copies of its data, both for data protection and tocomply with applicable regulatory requirements. These backup operationsmay be performed periodically. For example, in some instances, anorganization may backup the data for one or more of its data storesevery day, every week, or at any other suitable time interval, asdesired. Various techniques for utilizing a cloud-based data protectionand recovery service to perform data backup operations and other relatedoperations are described in detail in U.S. application Ser. No.16/857,007 (Attorney Docket No. 7251-00101) entitled “Cloud Based DataProtection Service” (hereinafter the “'007 Application”), which wasfiled on Apr. 23, 2020 and is hereby incorporated by reference as ifentirely set forth herein.

Though effective for OLTP, the data stores used by organizations aretypically not optimized for online analytical processing (“OLAP”)operations, which may be performed to gain insight based on the datawithin the system. For example, common data stores used for OLTPoperations (such as Oracle™ databases, etc.) are not designed for thecomplex queries typically utilized for OLAP operations, and it can takean extended period of time (e.g., hours, days, etc.) and significantprocessing resources to execute an OLAP query on an OLTP data store.Additionally, since an organization's data is commonly maintained in“data silos” (in which some of the organization's data is isolated from,and thus not accessible to, other parts of the organization's systems),the data needed for a given analytical job may be maintained acrossvarious, disparate data stores. Accordingly, when an organization needsto perform data analytics across such data silos, it is common toutilize a separate data warehousing system.

As used herein, the term “data warehouse” or “data warehousing system”refers a repository of data from one or more data sources that is usedprimarily for reporting and data analysis operations. Data warehousesare typically designed to perform OLAP operations characterized by theuse of potentially complex queries and the relatively low volume oftransactions (when compared to the number of transactions performed bythe organization's OLTP operations). To utilize a data warehouse,organizations traditionally extract data from their various data stores,transform the format and structure of the data as needed to fit thetarget data warehouse, and load that transformed data into the datawarehouse. This process is referred to as Extract-Transform-Load(“ETL”). Utilizing a data warehouse allows an organization to create adata view across data stores for a specific point-in-time. For example,performing this ETL process allows an organization to create apoint-in-time representation of its data so that it can perform dataanalysis across data stores. Such an approach presents various technicalshortcomings. For example, the ETL process is both time-intensive andcomputationally expensive. Additionally, since the ETL process resultsin a point-in-time view of the organization's data, this ETL processmust be performed each time the organization wishes to obtain adifferent view of its data (e.g., to perform additional analyticaloperations), resulting in significant expense.

To facilitate these backup and ETL operations, an organization istypically required to run, on its production servers (e.g., servers usedto host its business application(s)), software to perform the necessaryprocessing (e.g., generating the backup copies, performing thetransformations, etc.). Using the production servers for this purpose,however, negatively impacts the performance of the organization bywasting computational resources that could otherwise be used to providethe organization's business application services. Additionally, by doingboth backup and ETL operations, two independent workflows are beingperformed for the organization's data stores on a recurrent basis (e.g.,every day), hitting the production servers multiple times a day andfurther exacerbating the negative performance impact.

In various embodiments, the disclosed systems and methods solve theseand other technical problems by enhancing an organization's backup data(e.g., from multiple different data stores) with useful metadata andstoring the backup data, along with the metadata, in a time-series datalake. As used in the art, a “data lake” generally refers to a datarepository that is capable of storing structured data, semi-structureddata, unstructured data, or binary data. Accordingly, a data lake mightstore database records (structured data), XML, or JSON data(semi-structured data), emails or PDFs (unstructured data), images orvideo (binary data), etc. The present inventors have recognized thatenriching the data stored in a data lake with timestamp metadataprovides numerous technical improvements over prior data lakeimplementations. These data lakes may be referred to as “time-seriesdata lakes.” As used herein, the term “time-series data lake” refers toa data repository that stores a time-series representation of data (thatis, data that has been “enriched” to include one or more items oftimestamp metadata). Throughout the remainder of this disclosure,references to a “data lake” (such as data lake 120 of FIG. 1 below) areunderstood to refer to a time-series data lake, unless otherwise noted.Should the Applicant intend to refer to prior data lake implementations,the term “prior data lake” will be used.

As a non-limiting example, the metadata may include a timestamp of whenthe backup data was extracted from the organization's data store, anidentifier associated with the data store, a schema associated with thedata store, access control information indicating those users authorizedto access the source data, tags associated with the objects, filesystem-level permissions, record-level security (if applicable), thenature or classification of each column, object-level timestamps, file-or record-level checksums or hashed fingerprints, etc.

The disclosed data lake service may store the organization's backup datain a time-series data lake, for example using a column-oriented formatfor storing data (such as Apache™ Parquet, Apache™ Avro, Apache™ ORC, ora Clumio™-specific column-oriented format, as non-limiting examples),which may facilitate efficient retrieval of the organization's data. Forexample, in various embodiments, storing an organization's backup data,with this embedded metadata, in a time-series data lake may greatlyimprove the ability of the organization to retrieve a specific data view(e.g., across multiple data sources over a desired time period or at adesired point-in-time) without requiring the organization to performadditional, resource-intensive ETL operations. The term “view” (or “dataview”), as used herein, refers to a selection of a particular set ofdata, from one or more of an organization's data sources, based on oneor more criteria. For example, as described in greater detail below, thedisclosed techniques allow a user associated with (or with thepermission of) an organization to retrieve a desired view of theorganization's backup data from the time-series data lake, for examplebased on one or more search criteria (or queries) provided by the user.Non-limiting examples of criteria that may be used to retrieve a dataview from a time-series data lake include one or more of a timestamp, anidentifier associated with the data store, a schema associated with thedata store, access control information indicating those users authorizedto access the source data, tags associated with the objects, filesystem-level permissions, record-level security (if applicable), thenature or classification of each column, object-level timestamps, file-or record-level checksums or hashed fingerprints, etc. Since thedisclosed techniques utilize a time-series data lake in which theorganization's backup data, ingested into the data lake from multipledifferent data sources, is enriched with metadata and stored in the datalake, the “view” provided to the user may include data from multipledifferent (and, potentially, siloed) data sources and from multipledifferent points in time. As a non-limiting example, a data view of anorganization's backup data provided using the disclosed techniques mayinclude one or more items of data from a first data source from a backupperformed at a first point in time, one or more items of data from asecond data source from a backup performed at the first point in time,one or more items of data from the first data source from a backupperformed at a second point in time, one or more items of data from athird data source from a backup performed at a third point in time, etc.Accordingly, in various embodiments, the disclosed techniques enable anorganization to obtain a desired view of its backup data across itsvarious different data sources and from multiple different points intime.

The disclosed systems and methods may provide technical benefits invarious contexts. For example, in the context of data warehousing,various disclosed methods reduce the burden on an organization'sproduction data sources and hosts by eliminating the need for thedisparate systems and processes to perform the compute- andI/O-intensive ETL operations, instead retrieving the organization's dataas part of its data backup operations. Additionally, unlike traditionalETL operations, the disclosed systems and methods allow an organizationthe flexibility to generate data views across both data stores and timedimensions without the need to re-perform the ETL operations. Further,as described in more detail below, the disclosed systems and methods mayimprove an organization's ability to perform data retention (e.g., tocomply with a legal hold request) and to modify (e.g., delete, mask,anonymize, etc.) selected subsets of an organization's backup data(e.g., to comply with a “Right to Be Forgotten” request) in a fast,computationally efficient manner.

In FIG. 1, block diagram 100 depicts a data protection service 102,which includes a data lake service 110 and a time-series data lake 120.In various embodiments, the data protection service 102 is configured tostore backup data for various organizations. For example, in theembodiment of FIG. 1, a first organization (Org1) has two physical sites130 and 132. In FIG. 1, Org1 site 130 includes various hosts 140A-140N(e.g., server systems) and various data stores 144A-144N. Note that, invarious embodiments, the data stores 144 may be implemented using any ofvarious, and potentially different, data storage technologies, each ofwhich may utilize its own data storage format. As a non-limitingexample, data store 144A may be an Oracle™ database and data store 144B(not separately shown) may be an IBM™ Db2 database.

In FIG. 1, the various data stores 144 are stored on data storage device142. Note that, although a single data storage device 142 is shown inFIG. 1 for clarity, an organization may use any suitable number of datastorage devices 142. Data stores 144 may be stored using any suitabletype of data storage devices 142. The storage devices 142 may includeany type of non-transitory computer data storage that isreadable/writable. For example, the storage devices 142 may include oneor more disk drives. The disk drives may be magnetically based drives(e.g. “hard drives”), optically based drives such as compact disk ordigital video disk drives, solid state drives formed from various formsof integrated circuit based non-volatile memory (e.g. flash memory),etc. The storage devices 142 may include other forms of memory,including volatile memory such as random-access memory (staticrandom-access memory or “SRAM”, dynamic random-access memory or “DRAM”,etc.). Any type or combination of storage devices may be used in variousembodiments. Additionally (or in the alternative), Org1 may have a cloudaccount within a public or private cloud, which may use cloud computingresources to execute various applications and store data for Org1,rather than using on-site resources.

FIG. 1 further depicts a site 134 for a second organization (Org2).Sites 132 and 134 have been simplified in FIG. 1, for clarity. Notethat, in various embodiments, an organization's site (e.g., Org1 site132, Org2 site 134, etc.) may include any suitable number of data stores144. Further, note that, in various embodiments, an organization mayinclude one or more data stores that do not utilize the various servicesprovided by the data protection service 102. In such embodiments, thedata from such unprotected data stores may not be backed up (e.g., bythe data protection service 102) and may not be available in the datalake 120 for subsequent access by a requesting entity 160.

Data protection service 102 may provide cloud-based data backup servicesfor various organizations. For example, as shown in FIG. 1, the dataprotection service 102 may employ one or more local agents 150 in anorganization's systems (e.g., in Org1 site 130 or 132 or Org1'scloud-based resources). As described in greater detail in the '007Application, the local agent 150 may interact with the data protectionservice 102 to implement the data protections supplied by the dataprotection service 102. More specifically, the local agent 150, invarious embodiments, is operable to determine which blocks of data (inone or more data stores 144) have changed since the most recent backup,compress and encrypt the blocks, assemble the blocks in a format used bythe data protection service 102, and send the data (either directly orvia a cloud-based data storage system) to the data protection service102. Thus, rather than requiring the organization to hit each data store144 individually as part of a separate ETL operation, the disclosedsystems and methods use the data protection service 102 to perform theextraction operations on the organization's data stores 144 as part ofthe existing backup operations. That is, the data protection service 102is already extracting data from an organization's data stores 144 aspart of delivering backup and recovery services, in various embodiments,the disclosed techniques eliminate the need to burden the organization'sdata stores 114 twice—once for performing backup operations and againfor performing ETL operations. Accordingly, in the depicted embodiment,the data protection service 102 may receive backup data, sent by thelocal agent 150, from various data stores 144 of Org1.

Note that, from the perspective of the data protection service 102, adata store may be viewed as a “data source.” This convention ofreferring to a data store (e.g., data store 144A) as a “data source”when described from the perspective of the data protection service 102is used throughout the remainder of this disclosure. Further note that,in various embodiments, each of the data stores 144 for an organizationmay be backed up (e.g., backup data may be sent to data protectionservice 102) either at the same or different times. For example, in someembodiments, backup data for all of the data stores 144 may be sent atthe same time (e.g., during a non-peak time period for the organization)to the data protection service 102. In other embodiments, however, datafrom different data stores 144 may be backed up at different points intime (e.g., according to a schedule, upon satisfaction of one or morecriteria, etc.). Thus, when backup data is received at the dataprotection service 102, the backup data is associated with both a time(e.g., the time at which the backup data was sent to, or received at,the data protection service 102) and a source (e.g., the data store 144from which the backup data originated).

In some embodiments, the backup data may be provided in the form of a“physical backup” (also referred to herein as a “backup image” or a“snapshot”) which may include a copy of the physical files of a datastore (e.g., data files, log files, control files, etc.). Physicalbackups are often used, for example, in disaster recovery scenarios torecover a data store. A physical backup of a particular data store istypically provided in a format that is specific to the particular datastore. For example, in instances in which a data store 144 is an Oracle™database, a physical backup of that data store 144 is an Oracle™ RMANbackup. As another non-limiting example, in instances in which a datastore 144 is an Amazon™ AWS™ RDS instance, the physical backup is an RDSsnapshot of the RDS instance. In prior approaches, a backup system maysimply store these backup images of the data stores 144. To subsequentlyaccess the data in these backup images, the backup images must berestored in the format in which they were originally saved. As anon-limiting example, to access data from a backup image of an Oracle™database, one would need to restore that image using the Oracle™ RDBMSsoftware. Such an approach presents various technical shortcomings. Forexample, this approach severely limits the ability of this backup datato be accessed in an on-demand manner. In such a system, for example, itwould not be feasible or desirable to create custom data views of datafrom multiple data sources or from multiple points in time because, todo so, the system would have to restore all of the backup images ontodedicated sandbox environments that are equipped with the original datastore software, and scan the regenerated data sources end-to-end toidentify relevant records for OLAP processing, wasting a significantamount of time and computational resources.

In various embodiments, however, the disclosed data protection service102 may extract the backup data from these backup images and enhance theextracted data with metadata (e.g., a timestamp associated with theextraction job, data source of the data, database schema of the datasource, authentication and authorization information, access controllists, tags associated with the objects, file system-level permissions,record-level security (if applicable), the nature or classification ofeach column, object-level timestamps, file- or record-level checksums orhashed fingerprints, etc.). For example, in the depicted embodiment,data lake service 110 includes conversion module 112 and metadataenrichment module 114. Conversion module 112, in various embodiments, isoperable to convert a physical backup of a data store 144 from a formatassociated with that data store 144 (e.g., from an RDS snapshot format,Oracle™ RMAN backup format, etc.) into a “logical backup” of that datastore 144. For example, in some embodiments, creating a logical backupfrom a physical backup includes using vendor-specific APIs to extractdata from its original, proprietary format to general purpose, readablefiles and records that may be ingested into the data lake 120 with oneor more items of metadata.

As will be appreciated by one of skill in the art with the benefit ofthis disclosure, the term “logical backup” refers to an extracted copyof the logical data elements (e.g., tables, records, metadata, etc.)from a data source. As used herein, the term “data element” refers to alogical data representation that is no longer tied to the (potentiallyproprietary) format of the original data source from which the dataelement was extracted. Whereas the physical backup of a data source tiesthe data to a format that is proprietary to the data source's managementsoftware (e.g., Oracle™ RDBMS), the logical backup acts as theextraction mechanism where data is decoupled from backend softwareinfrastructure of the source and is infused with metadata, by themetadata enrichment module 114, before it is stored in data lake 120.For example, in embodiments in which a particular data store 144 is anOracle™ database, the physical backup of this data store 144 may be anOracle™ RMAN backup, from which the conversion module 112 may extractvarious different types of data elements, including records, tables,indexes, etc. that are no longer tied to the proprietary Oracle™database format. Thus, in various embodiments, the logical backup beinggenerated has no dependency on the original data source software and ispassed through a columnar compression engine after metadata enrichment.As will be appreciated by one of skill in the art with the benefit ofthis disclosure, storing the data elements in a column-oriented formatmay provide various technical benefits, such as optimizing the storageformat for faster queries, providing granular retrieval, and enablinglarger scale. Continuing with the example above, for instance, in whicha data store 144 is an Oracle™ database, the conversion module 112 mayconvert the physical backup (the Oracle™ RMAN backup) into a logicalbackup that extracts all of the data records that were in the Oracle™database at the time the Oracle™ RMAN backup was created. These recordsare metadata enriched to include metadata such as source identifier,time marker, authentication and authorization and so on before it landson columnar compression layer of the data lake 120.

By converting backup data from a backup image to a logical backup, thedata lake service 110 is able to enrich the backup data with valuablemetadata and store that enriched backup data in a time-series data lake120 in a format that allows for efficient retrieval (e.g., Apache™Parquet format), as described in more detail below. For example,metadata enrichment module 114, in various embodiments, is operable togenerate metadata associated with the backup and embed the metadata intothe logical backup data. Non-limiting examples of items of metadata thatmay be generated by metadata enrichment module 114 include: a timestampof when the backup data was extracted from the organization's data store144 or received at the data protection service 102, an identifierassociated with the data store 144 from which the backup dataoriginated, a schema associated with the data store 144, access controlinformation indicating those users and systems authorized to access thesource data, tags associated with the objects, file system-levelpermissions, record-level security (if applicable), the nature orclassification of each column, object-level timestamps, file- orrecord-level checksums or hashed fingerprints, etc.

Note that, in some embodiments, one or more items of metadata may beincluded with the backup data sent by the local agent 150. For example,in some embodiments, when local agent 150 sends a backup image of agiven data store 144 to the data protection service 102, the local agent150 may include various items of metadata, such as when the backup imagewas created, a schema associated with the data store 144, access controlinformation associated with the data store 144 (or the data records ortables contained therein), etc. In some such embodiments, the localagent 150 may retrieve one or more items of metadata from the data store144 as it prepares the backup data to be sent to the data protectionservice 102. Accordingly, in some embodiments, metadata enrichmentmodule 114 is operable to “generate” metadata corresponding to a backupoperation by identifying the one or more items of metadata sent alongwith the backup image of a data source 144. Additionally, in someembodiments, metadata enrichment module 114 is operable to generateitems of metadata locally. For example, in some embodiments, metadataenrichment module 114 is operable to generate a timestamp associatedwith backup data in response to receiving the backup data from the localagent 150. As a non-limiting example, in embodiments that utilize theJava™ programming language, metadata enrichment module 114 may usemethods from the java.util.Date or java.util.Calendar classes togenerate a timestamp that corresponds to the time at which backup datafor a particular data store 144 was received by the data protectionservice 102. Further note that, in some instances, a given item ofmetadata may correspond to backup data for an entire data store 144(e.g., such as a timestamp corresponding to the time at which a backupimage was generated). In other instances, however, a given item ofmetadata may correspond to a subset of backup data for a data store 144.For example, in many instances, a data store 144 may be a database thatincludes numerous data records, each which may have a field withcorresponding access control information indicating those users who are(or are not) permitted to access the data in that record. In someembodiments, such access control information may be considered metadatathat is specific to the individual data records within a data store 144.

Data lake service 110, in various embodiments, is operable to store thelogical backup data for an organization (Org1, in the current example),along with the generated metadata, in a data lake 120 associated withthe organization. Stated differently, in various embodiments, the datalake service 110 is operable to transform an organization's backup datainto a logical, addressable format that is enriched with various itemsof metadata, such as timeline information, access control information,and source information. Thus, in various embodiments, a data lake 120provides a time-series representation of various data sources 144 in anorganization's system. As described in more detail below, in variousembodiments, the metadata enrichment process is performed duringingestion of data onto a data lake 120 to create the time-seriesrepresentation of one or more of an organization's data sources. Notethat, in some embodiments, data may be stored in a data lake 120 as aself-describing object, rather than as a data record. For example, asnoted above, in some embodiments, the data lake service 110 stores thisbackup data in the data lake 120 using the Apache™ Parquet format.

In FIG. 1, data protection service 102 includes a data lake storagesystem 118, which stores various data lakes 120A-120N. In variousembodiments, data lake storage system 118 may be implemented using thesimple storage service (“S3”) in the AWS™ public cloud. Note, however,that this embodiment is provided merely as one non-limiting example and,in other embodiments, any suitable data storage system may be used. Insome embodiments, each organization may have its own data lake 120 usedto store the time-series data for the organization. In the depictedembodiment, for example, data protection service 102 maintains a datalake 120A that stores “enriched” time-series data (that is, logical datawith embedded metadata, such as timing information) for Org1. In someembodiments, data lake service 110 may maintain multiple data lakes 120for an organization. Note that, in various embodiments, data lakeservice 110 may store time-series data for an organization (e.g., Org1)from multiple different physical sites (e.g., sites 130 and 132) andcloud-based sites in a single data lake 120 (e.g., data lake 120A). Invarious embodiments, this technique enables the data protection service102 to provide visibility to the user(s) in an organization of acrossall data stores of the organization, regardless of the particularlocation at which data happens to be stored.

Note that, as used herein, the term “time-series data lake” differs froma “time-series database.” A time-series database is a software systemthat is optimized for storing and serving time-series data throughassociated pairs of times and values. For example, early time-seriesdatabases were associated with industrial applications that could storemeasured values from sensory equipment and are now used in support of awider range of applications in production systems. The disclosedtime-series data lake 120, unlike a time-series database, is not adatabase and, as such, is not designed to record time-series data in astructured manner from sensors or other front-end applications. Instead,in various embodiments, the disclosed time-series data lake sources datafrom pre-existing data sources (which, in some embodiments, may includeone or more time-series databases) and liberates that data from theproprietary format of the original data sources and enriches that datawith one or more items of metadata.

Data lake service 110 further includes data retrieval module 116, which,in various embodiments, is operable to retrieve a selected view of anorganization's time-series data, across one or more sources, from thedata lake 120. For example, in the depicted embodiment, a requestingentity 160 sends a request 162 for a particular view of anorganization's data. In some embodiments, the requesting entity 160 maybe a data warehousing system (or a user associated with a datawarehousing system) requesting, for example, a particular view of Org1'stime-series data from the data lake 120A. Some such embodiments aredescribed in more detail below with reference to FIG. 6. In otherembodiments, the requesting entity 160 may be a user (e.g., a datascientist) associated with an organization that is directly connectingto a data lake 120. For example, in some embodiments, data retrievalmodule 116 operates as a driver (an Open Database Connectivity (“ODBC”)protocol driver) that allows a requesting entity (e.g., a data scientistusing a business intelligence tool, such as Tableau™) to directly accessand query a data lake 120. As a non-limiting example of one suchembodiment, the requesting entity 160 may be a data scientist associatedwith Org1 that is directly connecting to Org1's data lake 120A toperform an ODBC query (e.g., as an ad hoc query rather than a completedata warehousing operation) against Org1's time-series data maintainedin the data lake 120A.

In various embodiments, the request 162 may include one or more criteriathat the data retrieval module 116 may use to query the data lake 120Afor the requested data view 164. As non-limiting examples, theparameters in the request 162 may include a point-in-time for which toretrieve data, a time period for which to retrieve data, an identifierof the organization for which to retrieve data, an identifier of thedata lake 120 from which to retrieve data, authentication orauthorization information (e.g., a token, credentials, etc.) associatedwith the organization or data lake 120, one or more data sources 144from which to retrieve data, one or more search terms or identifiers(e.g., alphanumeric keywords) to include in the query, American NationalStandards Institute (“ANSI”) SQL-based queries, or any other suitablecriteria that the data retrieval module 116 may use to retrieve thedesired data view 164 from the data lake 120A. In various embodiments,the data retrieval module 116 is operable to construct a query based onthese criteria and execute the query against the data lake 120A. Thus,in various embodiments, the data retrieval module 116 is operable tosearch across multiple, disparate data sources (e.g., all of, or anycombination of, data stores 144A-144N) and times, using the embeddedmetadata in the data lake 120A to retrieve the desired data view in afast and computationally efficient manner. In some embodiments, the dataretrieval module 116 operates as a retrieval service that parses dataand metadata in the data lake 120 (e.g., based on one or more criteria)to retrieve desired data views 164 from an organization's backup datastored in the data lake 120. These desired data views may then beprovided to a requesting entity 160 (e.g., a data warehousing system),as desired. Further, in some embodiments, the data retrieval module 116is operable to serve as a driver (e.g., an ODBC driver) that allows arequesting entity (such as business intelligence tools, external tablesinterface in a data warehouse, etc.) to directly query a data lake 120.

In various embodiments, the disclosed systems and methods allow anorganization to generate, at least, the following data views of theorganization's data in an on-demand manner in the cloud, withoutrequiring the organization to perform additional ETL operations usingits production resources:

-   -   1. A view of all data stored in the data lake 120 for the entire        time-series as a single data view;    -   2. A view of all data stored in the data lake 120 for a selected        time window in the time-series;    -   3. A view of all data stored in the data lake 120 for a selected        point-in-time;    -   4. A view of data from a specific data source (e.g., data store        152A) in the data lake 120 for the entire time series;    -   5. A view of data from a specific data source in the data lake        120 for a selected time window in the time-series;    -   6. A view of data from a specific data source in the data lake        120 for a selected point-in-time;    -   7. A view of a subset of the data from a specific data source in        the data lake 120 for the entire time-series;    -   8. A view of a subset of the data from a specific data source in        the data lake 120 for a selected time window in the time-series;    -   9. A view of a subset of the data from a specific data source in        the data lake 120 for a selected point-in-time;    -   10. A view of a union of subsets of data stored in the data lake        120, from any number of data sources, for the entire        time-series;    -   11. A view of a union of subsets of data stored in the data lake        120, from any number of data sources, for a selected time window        in the time-series; or    -   12. A view of a union of subsets of data stored in the data lake        120, from any number of data sources, for a selected        point-in-time

The disclosed systems and methods may allow retrieval of any of thesevarious data views in an on-demand manner for various purposes. Forexample, as noted above, an organization may utilize a data warehouse toperform analytical operations on selected view of data from the datalake 120. Some such embodiments are described in more detail below withreference to FIG. 6. Additionally, note that although data lake service110 has been described as retrieving data views for a data warehousingsystem, the present disclosure is not limiting to such embodiments. Forexample, in some embodiments, the disclosed systems and methods may beused to perform data preservation operations to preserve subsets of dataincluded in a data lake 120 (described below with reference to FIGS.6-7) and to modify (e.g., delete, mask, anonymize, etc.) data stored inan organization's data lake 120 in an efficient manner (described belowwith reference to FIGS. 8-9).

Turning now to FIG. 2, block diagram 200 depicts data protection service102 in more detail, according to some embodiments. More specifically, inthe embodiment of FIG. 2, data protection service 102 includes backupservice 202, which, in various embodiments, is operable to perform oneor more of the various data protection services described in the '007Application. For example, in some embodiments, backup service 202 is (oris included as part of) the Clumio™ backup and data protection servicethat provides secure backup-as-a-service. In the depicted embodiment,for example, backup service 202 is operable to store backup data 204 forone or more organizations in one or more data storage devices 206. ForOrg1 of FIG. 1, for instance, backup service 202 is operable to receivebackup data (e.g., as backup images) corresponding to Org1's variousdata stores 144. For example, as described in more detail in the '007Application, one or more of Org1's hosts 140 may include a local agent150 that is operable to periodically provide backup data from thevarious data stores 144 to the data protection service 102, which maythen store the backup data 204 on one or more data storage devices 206.Note that, although shown as part of data protection service 102 in FIG.1, backup service 202 may store backup data 204 on one or more local orremote data storage devices 206, according to various embodiments. As anon-limiting example, data storage device 206 may be implemented asblock storage supported by one or more public cloud services (e.g.,elastic block store (“EBS”) in the AWS™ public cloud, as onenon-limiting example). The block storage may support a block-basedstorage and access mechanism, similar to the access model for diskdrives and the like. In other embodiments, data storage device 208 maybe implemented using an object-based storage supported by one or morepublic cloud services (e.g., S3 in the AWS™ public cloud). Further notethat, in various embodiments, data protection service 102 is implementedas a cloud-based service, utilizing public or private cloud-basedresources, such as server computer systems, data stores, data storagedevices, networking elements, etc., to perform the various operationsdescribed herein. As one non-limiting example, in some embodiments, thedata protection service 102 may be implemented using various servicesprovided by the AWS™ cloud-computing platform.

In various embodiments, backup data for an organization may be providedto the data protection service 102 in various formats. For example, FIG.3A depicts an example embodiment in which the backup data for Org1 isprovided in the form of physical backups of one or more of Org1's datastores 144. In such embodiments, data lake service 110 is operable toconvert the physical backups into logical backups, extracting thelogical data elements (e.g., tables, records, etc.) and enhancing theextracted data with various items of metadata and storing the enrichedbackup data in the data lake 120A, as described above. In someembodiments, however, note that the backup data may be provided to thedata protection service 102 already in a logical backup format. Forexample, FIG. 3B depicts an example embodiment in which the backup datafor Org1 is provided in the form of logical backups of one or more ofOrg1's data stores 144. In such embodiments, data lake service 110 isoperable to generate metadata associated with this backup data and storethe enriched backup data in the data lake 120A, as described above.

Referring now to FIG. 4A-4B, flow diagrams illustrating example methods400 and 450 for storing backup data, along with one or more items ofmetadata, in a time-series data lake are respectively depicted,according to two non-limiting embodiments. In various embodiments,method 400 or 450 may be performed by data protection service 102 ofFIG. 1 to store backup data for Org1, from one or more data stores 144,in data lake 120A. As noted above, in various embodiments, dataprotection service 102 may be implemented as a cloud-based service usingpublic or private cloud-based computing resources, such as servercomputer systems. In some such embodiments, the server computersystem(s) used to implement the data protection service 102 may include(or have access to) a non-transitory, computer-readable medium havingprogram instructions stored thereon that are executable by the servercomputer system(s) to cause the operations described with reference toFIG. 4A or 4 b. In FIG. 4A, method 400 includes elements 402-408. InFIG. 4B, method 450 includes elements 452-458. While these elements areshown in a particular order for ease of understanding, other orders maybe used. In various embodiments, some of the method elements may beperformed concurrently, in a different order than shown, or may beomitted. Additional method elements may also be performed as desired.

At 402, in the illustrated embodiment, the data protection service 102receives backup data for an organization as part of one or more backupoperations. In some embodiments, the backup data includes a first backupimage, of a first data source, that was generated at a first point intime. For example, as shown in FIG. 3A, data protection service 102 mayreceive (e.g., from a local agent 150 executing on one or more hosts 140at Org1 site 130) a physical backup of a data source 144A that wascreated at a first point in time t₁. At 404, in the illustratedembodiment, the data protection service 102 creates a logical backup ofthe first data source using the first backup image. For example, in someembodiments, the conversion module 112 is operable to convert a physicalbackup (e.g., a backup image) of a data source into a logical backup ofthat data source. In some embodiments, the logical backup includes aplurality of data records from the first data source at the first pointin time. For example, in some embodiments, creating the logical backupof the first data source may include extracting the plurality of datarecords from the first backup image such that logical backup data in thelogical backup is independent of a physical backup format associatedwith the first data source.

At 406, in the illustrated embodiment, the data protection service 102generates metadata associated with the data source. For example,metadata enrichment module 114 may generate metadata associated withbackup data from a data source 144A that is received from the Org1. Asnon-limiting examples, the metadata may include timestamp informationcorresponding to the first point in time t₁ at which the physical backupwas made, an identifier associated with the first data source, accesscontrol information identifying users with access to one or more of theplurality of data records, a schema associated with the first datasource, etc. At 408, in the illustrated embodiment, the data protectionservice 102 stores the logical backup and the metadata in a time-seriesdata lake 120A associated with the organization. Note that, in someembodiments, method 400 may include the data protection service 102embedding the metadata into the logical backup. For example, in variousembodiments, the metadata enrichment module 114 is operable to embed anorganization's backup data with metadata (e.g., source and timestampinformation) before storing the enriched backup data in the data lake120. In various embodiments, 408 may include storing the logical backupdata, with the embedded metadata, in the time-series data lake 120A in acolumn-oriented format (e.g., Apache™ Parquet).

Turning now to FIG. 4B, method 450 begins at 452 where, in theillustrated embodiment, the data protection service 102 provides a datalake service 110 that maintains data for a plurality of organizations.For example, for a first organization (Org1), the data lake service 110may maintain a time-series data lake that stores a time-seriesrepresentation of data associated with the first organization.

At 454, in the illustrate embodiment, the data protection service 102receives backup data from a plurality of data sources associated withthe first organization. The backup data, in some embodiments, mayinclude a first physical backup, of a first one of the plurality of datasources, which was created at a first point in time and provided in afirst format. As non-limiting examples, the first physical backup may bean Amazon™ RDS snapshot, a snapshot of an Amazon™ DynamoDB database, asnapshot of an Amazon™ Neptune database, a snapshot of an Amazon™ Auroradatabase, a snapshot of a Microsoft™ Azure SQL database, etc., accordingto various embodiments. Further, in some embodiments, the backup datamay include a second physical backup, of a second one of the pluralityof data sources, which was created at a second point in time andprovided in a second, different format. Note that, in variousembodiments, the backup data may include structured data (e.g.,databases), semi-structured data (e.g., XML or JSON data), orunstructured data (e.g., emails, PDFs, etc.), or binary data (e.g.,images, video, audio, etc.).

In some embodiments, method 450 may include the data protection service102 converting the backup data from one or more physical backup formatsinto logical backup data. For example, in some embodiments, convertingthe backup data from one or more physical backup formats into logicalbackup data includes converting the first physical backup into a firstlogical backup of the first data source, where the first logical backupincludes a first plurality of data records from the first data source atthe first point in time. Further, in some embodiments, converting thebackup data may include converting the second physical backup into asecond logical backup of the second data source, where the secondlogical backup includes a second plurality of data records from thesecond data source at the second point in time.

At 456, in the illustrated embodiment, the data protection service 102generates metadata associated with the backup data received at element454. For example, for a given data element of the backup data, thecorresponding may include source information identifying a particularone of the plurality of data sources from which the given data elementof the backup data originated. At 458, in the illustrated embodiment,the data protection service 102 stores the backup data and the metadatain the time-series data lake 120A. For example, 458 may includeembedding the metadata into the logical backup data and then storing thelogical backup data, with the embedded metadata, in the time-series datalake 120A using a particular file format (e.g., Apache™ Parquet format).

Referring now to FIG. 5, a flow diagram illustrating an example method500 for retrieving data from a time-series data lake is depicted,according to some embodiments. In various embodiments, method 500 may beperformed by data protection service 102 of FIG. 1 to retrieve aparticular view of backup data for a particular organization (e.g.,Org1) from a data lake 120 (e.g., data lake 120A) maintained by the dataprotection service 102. As noted above, in various embodiments, dataprotection service 102 may be implemented as a cloud-based service usingpublic or private cloud-based computing resources, such as servercomputer systems. In some such embodiments, the server computersystem(s) used to implement the data protection service 102 may include(or have access to) a non-transitory, computer-readable medium havingprogram instructions stored thereon that are executable by the servercomputer system(s) to cause the operations described with reference toFIG. 5. In FIG. 5, method 500 includes elements 502-508. While theseelements are shown in a particular order for ease of understanding,other orders may be used. In various embodiments, some of the methodelements may be performed concurrently, in a different order than shown,or may be omitted. Additional method elements may also be performed asdesired.

At 502, in the illustrated embodiment, the data protection service 102provides a data lake service 110 that maintains data for a plurality oforganizations where, for a first organization (e.g., Org1), the datalake service 110 maintains a time-series data lake 120A that stores atime-series representation of backup data associated with the firstorganization. In some embodiments, the backup data may include firstbackup data from a first data source created at a first point in timeand second backup data from a second data source created at a second,different point in time. In some embodiments, the time-seriesrepresentation of the backup data includes metadata associated with thebackup data. For example, for a given data element of the backup data inthe time-series data lake 120A, the metadata may indicate a data sourcefrom which the given data element of the backup data originated and atime at which the given data element of the backup data was backed up.

At 504, in the illustrated embodiment, the data protection service 102receives, from a requesting entity, a request for data associated withthe first organization. In some embodiments, for example such as thosedescribed in more detail below with reference to FIGS. 6-7, therequesting entity may be (or be associated with) a data warehousingsystem. In other embodiments, for example such as those discussed inmore detail below with reference to FIGS. 8-9, the request received at504 may be sent from a data preservation service.

At 506, in the illustrated embodiment, based on one or more of thesearch criteria included in the request, the data protection service 102retrieves a particular view of the backup data stored in the time-seriesdata lake. In some embodiments, the retrieving the particular view ofthe backup data includes executing a query against the time-series datalake 120A, for example by parsing the metadata associated with thebackup data based on the one or more search criteria. Note that, in someembodiments, the request received at 504 may include a query specifiedby the requesting entity (e.g., provided using the Open DatabaseConnectivity (“ODBC”) protocol). In other embodiments, method 500 mayinclude creating a query based on the one or more of the search criteriaincluded in the request.

At 508, in the illustrated embodiment, the data protection service 102provides the particular view of the backup data to the requestingentity. In some embodiments, for example, the particular view of thebackup data may include a subset of data that was extracted from aparticular backup of a first data source. In some embodiments, method500 further includes storing, by the data-protection service, theparticular view of the backup data in a particular storage location thatretains data according to a first data-retention policy that differsfrom a second data-retention policy of the time-series data lake 120A.In some such embodiments, subsequent to storing the particular view inthe particular storage location, method 500 includes the data protectionservice 102 monitoring the time-series data lake 120A to determinewhether updated backup data also matches the one or more search criteriaspecified in the request.

Example Embodiments: Storing Backup Data in a Time-Series Data Lake

-   1. A method, comprising:

providing, by a cloud-based service, a data lake service that maintainsdata for a plurality of organizations, wherein, for a first one of theplurality of organizations, the data lake service maintains atime-series data lake that stores a time-series representation of dataassociated with the first organization;

receiving, by the cloud-based service, backup data, associated with thefirst organization, from a plurality of data sources;

generating, by the cloud-based service, metadata associated with thebackup data, wherein, for a given data element of the backup data,corresponding metadata includes source information identifying aparticular one of the plurality of data sources from which the givendata element of the backup data originated; and

storing, by the cloud-based service, the backup data and the metadata inthe time-series data lake.

-   2. The method of claim 1, wherein the backup data includes:

a first physical backup, created at a first point in time, of a firstone of the plurality of data sources, wherein the first physical backupis provided in a first format; and

a second physical backup, created at a second point in time, of a secondone of the plurality of data sources, wherein the second physical backupis provided in a second format.

-   3. The method of claim 2, wherein the first physical backup is an    Amazon™ RDS Snapshot.-   4. The method of claim 2, further comprising:

converting, by the cloud-based service, the backup data from one or morephysical backup formats, associated with the plurality of data sources,into logical backup data.

-   5. The method of claim 4, wherein the storing the backup data and    the metadata in the time-series data lake includes:

embedding the metadata into the logical backup data; and

storing the logical backup data, with the embedded metadata, in thetime-series data lake in a particular format.

-   6. The method of claim 5, wherein the particular format is the    Apache™ Parquet file format.-   7. The method of claim 4, wherein the converting the backup data    from the one or more physical backup formats into logical backup    data includes:

converting the first physical backup into a first logical backup of thefirst data source, wherein the first logical backup includes a firstplurality of data records from the first data source at the first pointin time; and

converting the second physical backup into a second logical backup ofthe second data source, wherein the second logical backup includes asecond plurality of data records from the second data source at thesecond point in time.

-   8. The method of claim 7, wherein, for a given one of the first    plurality of data records, the metadata includes a timestamp    indicative of the first point in time.-   9. The method of claim 7, wherein, for a given one of the first    plurality of data records, the metadata specifies the first data    source and the first point in time; and

wherein, for a given one of the second plurality of data records, themetadata specifies the second data source and the second point in time.

-   10. The method of claim 1, wherein the backup data includes    unstructured data.-   11. A non-transitory, computer-readable medium having program    instructions stored thereon that are executable by one or more    computer systems to perform operations comprising:

providing a cloud-based data lake service that maintains data for aplurality of organizations, wherein, for a first one of the plurality oforganizations, the cloud-based data lake service maintains a time-seriesdata lake that stores a time-series representation of data associatedwith the first organization;

receiving backup data, associated with the first organization, from aplurality of data sources;

generating metadata associated with the backup data, wherein, for agiven data element of the backup data, corresponding metadata includessource information identifying a particular one of the plurality of datasources from which the given data element of the backup data originated;and

storing the backup data and the metadata in the time-series data lake.

-   12. The non-transitory, computer-readable medium of claim 11,    wherein the backup data includes:

a first physical backup, created at a first point in time, of a firstone of the plurality of data sources, wherein the first physical backupis provided in a first format; and

a second physical backup, created at a second point in time, of a secondone of the plurality of data sources, wherein the second physical backupis provided in a second format.

-   13. The non-transitory, computer-readable medium of claim 12,    wherein the operations further comprise:

converting the backup data from one or more physical backup formats,associated with the plurality of data sources, into logical backup data.

-   14. The non-transitory, computer-readable medium of claim 13,    wherein the storing the backup data and the metadata in the    time-series data lake includes:

embedding the metadata into the logical backup data; and

storing the logical backup data, with the embedded metadata, in thetime-series data lake in a particular format.

-   15. The non-transitory, computer-readable medium of claim 13,    wherein the converting the backup data from the one or more physical    backup formats into logical backup data includes:

converting the first physical backup into a first logical backup of thefirst data source, wherein the first logical backup includes a firstplurality of data records from the first data source at the first pointin time; and

converting the second physical backup into a second logical backup ofthe second data source, wherein the second logical backup includes asecond plurality of data records from the second data source at thesecond point in time.

-   16. The non-transitory, computer-readable medium of claim 15,    wherein, for a given one of the first plurality of data records, the    metadata includes a timestamp indicative of the first point in time.-   17. A method, comprising:

receiving, by a cloud-based service, backup data for an organization aspart of one or more backup operations, wherein the backup data includesa first backup image, of a first data source, that was generated at afirst point in time;

creating, by the cloud-based service, a logical backup of the first datasource using the first backup image, wherein the logical backup includesa plurality of data records from the first data source at the firstpoint in time;

generating, by the cloud-based service, metadata associated with thefirst backup image; and

storing, by the cloud-based service, the logical backup and the metadatain a time-series data lake associated with the organization.

-   18. The method of claim 17, wherein the creating the logical backup    of the first data source includes extracting the plurality of data    records from the first backup image such that logical backup data in    the logical backup is independent of a physical backup format    associated with the first data source; and

wherein the method further comprises embedding the metadata into thelogical backup data.

-   19. The method of claim 18, wherein the storing the logical backup    and the metadata in the time-series data lake includes:

storing the logical backup data, with the embedded metadata, in thetime-series data lake in a column-oriented format.

-   20. The method of claim 17, wherein the metadata includes at least    one of:

a time stamp corresponding to the first point in time;

an identifier associated with the first data source;

access control information identifying users with access to one or moreof the plurality of data records; and

a schema associated with the first data source.

Example Embodiments: Retrieving Data from a Time-Series Data Lake

-   1. A method, comprising:

providing, by a cloud-based service, a data lake service that maintainsdata for a plurality of organizations, wherein, for a first one of theplurality of organizations, the data lake service maintains atime-series data lake that stores a time-series representation of backupdata associated with the first organization;

receiving, by the cloud-based service from a requesting entity, arequest for data associated with the first organization, wherein therequest includes one or more search criteria;

based on the one or more search criteria, retrieving, by the cloud-basedservice, a particular view of the backup data stored in the time-seriesdata lake; and

providing, by the cloud-based service, the particular view of the backupdata to the requesting entity.

-   2. The method of claim 1, wherein the retrieving the particular view    of the backup data includes executing a query against the    time-series data lake, wherein the query is based on the one or more    search criteria.-   3. The method of claim 1, wherein the backup data includes:

first backup data from a first data source, wherein the first backupdata was created at a first point in time; and

second backup data from a second data source, wherein the second backupdata was created at a second point in time; and

wherein the particular view of the backup data includes a subset of thefirst backup data and a subset of the second backup data.

-   4. The method of claim 1, wherein the time-series representation of    the backup data includes metadata associated with the backup data,    wherein, for a given data element of the backup data in the    time-series data lake, the metadata indicates:

a data source from which the given data element of the backup dataoriginated; and

a time at which the given data element of the backup data was backed up;and

wherein the retrieving the particular view includes parsing the metadataassociated with the backup data based on the one or more searchcriteria.

-   5. The method of claim 1, wherein the requesting entity is a data    warehousing system.-   6. The method of claim 1, wherein the request for data associated    with the first organization is received from a data preservation    service.-   7. The method of claim 1, wherein the particular view of the backup    data includes first data that was extracted from a particular backup    of a first data source, and wherein the particular view does not    include all of the data from the particular backup of the first data    source.-   8. The method of claim 7, further comprising:

storing, by the cloud-based service, the particular view of the backupdata in a particular storage location that retains data according to afirst retention policy that differs from a second retention policy ofthe time-series data lake.

-   9. The method of claim 8, further comprising:

subsequent to the storing the particular view in the particular storagelocation, monitoring, by the cloud-based service, the time-series datalake to determine whether updated backup data matches the one or moresearch criteria.

-   10. A non-transitory, computer-readable medium having instructions    stored thereon that are executable by a computer system to perform    operations comprising:

providing a cloud-based data lake service that maintains data for aplurality of organizations, wherein, for a first one of the plurality oforganizations, the cloud-based data lake service maintains a time-seriesdata lake that stores a time-series representation of backup dataassociated with the first organization;

receiving, from a requesting entity, a request for data associated withthe first organization, wherein the request includes one or more searchcriteria;

based on the one or more search criteria, retrieving a particular viewof the backup data stored in the time-series data lake; and

providing the particular view of the backup data to the requestingentity.

-   11. The non-transitory, computer-readable medium of claim 10,    wherein the retrieving the particular view of the backup data    includes executing a query against the time-series data lake,    wherein the query is based on the one or more search criteria.-   12. The non-transitory, computer-readable medium of claim 10,    wherein the backup data includes:

first backup data from a first data source, wherein the first backupdata was created at a first point in time; and

second backup data from a second data source, wherein the second backupdata was created at a second point in time; and

wherein the particular view of the backup data includes a subset of thefirst backup data and a subset of the second backup data.

-   13. The non-transitory, computer-readable medium of claim 10,    wherein the time-series representation of the backup data includes    metadata associated with the backup data, wherein, for a given data    element of the backup data in the time-series data lake, the    metadata indicates:

a data source from which the given data element of the backup dataoriginated; and

a time at which the given data element of the backup data was backed up;and

wherein the retrieving the particular view includes parsing the metadataassociated with the backup data based on the one or more searchcriteria.

-   14. The non-transitory, computer-readable medium of claim 10,    wherein the request includes a query specified by the requesting    entity.-   15. The non-transitory, computer-readable medium of claim 14,    wherein the query is provided, by the requesting entity, using the    Open Database Connectivity (“ODBC”) protocol.-   16. The non-transitory, computer-readable medium of claim 10,    wherein the operations further comprise:

creating a query based on one or more search criteria included in therequest; and

wherein the retrieving the particular view of the backup data includesexecuting the query against the time-series data lake.

-   17. A system, comprising:

at least one processor;

a non-transitory, computer-readable medium having instructions storedthereon that are executable by the at least one processor to cause thesystem to:

-   -   provide a cloud-based data lake service that maintains data for        a plurality of organizations, wherein, for a first one of the        plurality of organizations, the cloud-based data lake service        maintains a time-series data lake that stores a time-series        representation of backup data associated with the first        organization;    -   receive, from a requesting entity, a request for data associated        with the first organization, wherein the request includes one or        more search criteria;    -   based on the one or more search criteria, retrieve a particular        view of the backup data stored in the time-series data lake; and    -   provide the particular view of the backup data to the requesting        entity.

-   18. The system of claim 17, wherein the backup data includes:

first backup data from a first data source, wherein the first backupdata was created at a first point in time; and

second backup data from a second data source, wherein the second backupdata was created at a second point in time; and

wherein the particular view of the backup data includes a subset of thefirst backup data and a subset of the second backup data.

-   19. The system of claim 17, wherein the time-series representation    of the backup data includes metadata associated with the backup    data, wherein, for a given data element of the backup data in the    time-series data lake, the metadata indicates:

a data source from which the given data element of the backup dataoriginated; and

a time at which the given data element of the backup data was backed up;and

wherein retrieving the particular view includes parsing the metadataassociated with the backup data based on the one or more searchcriteria.

-   20. The system of claim 17, wherein the instructions are further    executable to cause the system to:

store the particular view of the backup data in a particular storagelocation that retains data according to a first retention policy thatdiffers from a second retention policy of the time-series data lake; and

subsequent to the storing the particular view in the particular storagelocation, monitor the time-series data lake to determine whether updatedbackup data matches the one or more search criteria.

Providing Data Views from a Time-Series Data Lake to a Data WarehousingSystem

In various embodiments, the disclosed systems and methods may be used toretrieve, in an on-demand manner, select views of an organization'sbackup data stored in a data lake 120. These selected data views canthen be provided to a data warehousing system for use in analyticaloperations for the organization, allowing the organization to performanalytical operations using data from any combination of anorganization's data sources and across any desired time period or at anydesired point in time.

Prior techniques for providing selected data views to a data warehousefor analytical operations present various technical problems. Forexample, as noted above, organizations are traditionally required toperform time-intensive and computationally expensive ETL operations toconstruct a desired view of their data each time the organization wishesto perform analytical operations using a data warehousing system. Usingthis prior approach, an organization extracts data from its various datastores, transforms the format and structure of the extracted data asneeded to fit the target data warehousing system, and loads theextracted, transformed data the into data warehousing system. Since theETL process results in a point-in-time view of the organization's data,this ETL process must be performed each time the organization wishes toobtain a different view of its data (e.g., to perform additionalanalytical operations), resulting in significant delay to perform theanalytical operations and wasting the organization's productionresources that could otherwise be used to provide the organization'sbusiness application services.

In various embodiments, however, the disclosed systems and methods solvethese technical problems presented by prior approaches. For example, invarious embodiments, the disclosed techniques eliminate the need for anorganization to perform these complex ETL operations to feed data fromtheir data stores into a data warehouse. As described above, forexample, an organization may use the data protection service 102 to backup its various on-premises or cloud-based data stores 144. In variousembodiments, in addition to providing various cloud-based dataprotection services, the disclosed data protection service 102 isoperable to store the organization's backup data in a time-series datalake 120. As described above, data lake service 110 may store logicalbackups (rather than backup images) of an organization's data stores144, enriched with metadata (e.g., timestamp information, access controlinformation, data source identifiers, etc.), in a data lake 120associated with the organization. In various embodiments, storing theenriched, time-series data in an organization's data lake 120 can bethought of as satisfying the “E” (extraction) and “L” (loading)functions of the ETL process, and retrieving a desired view of thebackup data from the data lake 120 may be thought of as performing the“T” (transformation) function. In various embodiments, the discloseddata protection service 102 enables such “transformations”—retrieving adesired data view from an organization's backup data in a data lake120—to be performed on-demand. By performing the “transformation”operations on-demand, the disclosed systems and methods may be thoughtof as changing the ordering of the operations from the traditional E-T-Lto E-L-T. Further, since the transformation is being performed by thecloud-based data protection service 102, an organization may retrieveany number of views of its backup data without further burdening itsproduction resources (e.g., hosts 140 or data stores 144).

For example, in FIG. 6, block diagram 600 depicts an embodiment of dataprotection service 102 that is operable to retrieve selected views fromthe data lake 120 and provide those data views to a data warehouse 602,according to some embodiments. In various embodiments, storing anorganization's enriched, time-series backup data in a data lake 120facilitates efficient retrieval of select data views (e.g., from variousdata sources at any point(s) in time) of the backup data in a mannerthat does not further tax the organization's production resources. Inthe depicted embodiment, for example, data warehouse 602 sends a query604 to data protection service 102 that includes one or more criteriafor the desired data view. (Note that, in various embodiments, the query604 may originate from some other requesting entity 160 rather than fromthe data warehouse 602 itself) In various embodiments, the query 604 mayspecify any suitable combination of criteria for a desired data view,such as a time period or periods, one or more points of time, one ormore data sources, keywords, access control information, etc. In variousembodiments, query 604 may be specified using any suitabledatabase-access protocol. As one non-limiting example, query 604 may bespecified using the ODBC protocol, which, as will be appreciated by oneof skill in the art, is a common API used to access database managementsystems. In some embodiments, it may be desirable to utilize a commonprotocol, such as the ODBC protocol, to facilitate integration withcommon data science tools, such as Tableau™. By accepting queries usingthe ODBC protocol, the disclosed data lake service 110, in someembodiments, allows data scientists to provide queries directly fromtheir existing analytics tools. Note, however, that this embodiment isprovided merely as a non-limiting example and, in other embodiments, anysuitable format may be used for query 604.

In various embodiments, once the data protection service 102 receivesthe query 604, it may be passed to the data retrieval module 116. Asnoted above, in various embodiments, data retrieval module 116 isoperable to retrieve data from a data lake 120 for an organization. Inthe depicted embodiment, for example, data retrieval module 116 receivesthe query 604 and is operable to search the data lake 120 for data to beincluded in the desired data view. For example, in various embodiments,the data retrieval module 116 is operable to identify relevant data forthe desired data view using the embedded metadata maintained, along withthe backup data itself, in the data lake 120. As shown in FIG. 6, thedata lake service 110 may then provide the desired data view to the datawarehouse 602. Once the desired data view is received at the datawarehouse 602, various data analytics tools may be used to furtheranalyze the data, as desired. In various embodiments, this process ofretrieving a desired data view across both data source- andtime-dimensions can be performed on-demand and as-needed to facilitatecontinued analytical operations. Thus, in various embodiments, thedisclosed time-series data lake 120 makes it possible to perform datatransformations in an on-demand manner during the runtime, therebyeliminating much of the sunk costs in compute and storage typicallyincurred in traditional ETL workflows. Further, in various embodiments,the data retrieval module 116 (and, more generally, the data lakeservice 110) has access to elastic compute and storage resources,allowing the time-series data lake 120 to be used as a truescale-on-demand transformation layer for calling applications.

As a non-limiting example, the disclosed systems and methods, in someembodiments, are operable to retrieve the following data views from adata lake 120 in an on-demand manner that does not require theorganization's production resources to perform additional ETLoperations:

-   -   1. A view of all data in a data lake 120 for the entire        time-series;    -   2. A view of all data in a data lake 120 for a selected time        period in the time-series;    -   3. A view of all data in a data lake 120 for a selected point in        time;    -   4. A view of data belonging to a specific data source 144 in a        data lake 120 for the entire time-series;    -   5. A view of data belonging to a specific data source 144 in the        data lake 120 for a selected time period in the time-series;    -   6. A view of data belonging to a specific data source 144 in the        data lake 120 for a selected point in time;    -   7. A view of a subset of data from a specific data source 144 in        the data lake 120 for the entire time-series;    -   8. A view of a subset of data from a specific data source 144 in        the data lake 120 for a selected time period in the time-series;    -   9. A view of a subset of data from a specific data source 144 in        the data lake 120 for a selected point in time;    -   10. A view of a union of subsets of data from any number of data        sources 144 in the data lake 120 for the entire time-series;    -   11. A view of a union of subsets of data from any number of data        sources 144 in the data lake 120 for a selected time period in        the time-series; and    -   12. A view of a union of subsets of data from any number of data        sources 144 in the data lake 120 for a selected point in time.

The disclosed systems and methods may provide various technicalbenefits, according to various embodiments. For example, in someembodiments, utilizing the data protection service 102 to retrievedesired data views from the data lake 120 in an on-demand manner allowsan organization to avoid burdening its production resources (e.g., hosts140, data stores 144, data storage devices 142, etc.) withcomputationally expensive and I/O-intensive ETL operations. Instead, bysimply utilizing the data protection service 102 to perform its databackup operations for data protection and compliance purposes, theorganization is also able to serve the same backup data (which, usingprior backup services, would be stored as backup images) foroperationally intensive data warehousing and analytics needs. Further,unlike traditional ETL operations, using the disclosed systems andmethods, an organization has the flexibility to generate data viewsacross both data source- and time-dimensions without the need tore-perform ETL operations on its production resources. For example, invarious embodiments, the disclosed data lake service 110 may use anorganization's enriched backup data stored in a data lake 120 to createdata views from various time periods or points in time that include datafrom any number of the organization's data stores 144 that are backed upusing the data lake service 110.

Referring now to FIG. 7, a flow diagram illustrating an example method700 for providing requested data views to a data warehousing system isdepicted, according to some embodiments. In various embodiments, method700 may be performed by data protection service 102 of FIG. 1 toretrieve a particular view of backup data for a particular organization(e.g., Org1) from a data lake 120 (e.g., data lake 120A) in an on-demandmanner for a requesting data warehouse 602. As noted above, in variousembodiments, data protection service 102 may be implemented as acloud-based service using public or private cloud-based computingresources, such as server computer systems. In some such embodiments,the server computer system(s) used to implement the data protectionservice 102 may include (or have access to) a non-transitory,computer-readable medium having program instructions stored thereon thatare executable by the server computer system(s) to cause the operationsdescribed with reference to FIG. 7. In FIG. 7, method 700 includeselements 702-714. While these elements are shown in a particular orderfor ease of understanding, other orders may be used. In variousembodiments, some of the method elements may be performed concurrently,in a different order than shown, or may be omitted. Additional methodelements may also be performed as desired.

At 702, in the illustrated embodiment, the data protection service 102provides a data lake service 110 that maintains, for a firstorganization, a time-series data lake 120A storing a time-seriesrepresentation of backup data from a plurality of data sourcesassociated with the first organization. In some embodiments, thetime-series representation of the backup data includes metadataassociated with the backup data. For example, for a given data elementstored in the data lake 120, the metadata may indicate a particular datasource from which the given data element originated and a time at whichthe given data element was backed up. At 704, in the illustratedembodiment, the data protection service 102 receives additional backupdata associated with the first organization as part of one or morebackup operations, where the additional backup data includes a firstbackup image, of a first data source, that was generated at a firstpoint in time. For example, as described above with reference to FIGS. 1and 3A, the data protection service 102 may receive a physical backup ofone or more of Org1's data sources 144 as part of a backup operation.

At 706, in the illustrated embodiment, the data protection service 102creates a first logical backup of the first data source using the firstbackup image. For example, as described above, conversion module 112 mayconvert a physical backup of a data store 144 from a format associatedwith that data store 144 into a logical backup of that data store. At708, in the illustrated embodiment, the data protection service 102stores the first logical backup, with corresponding metadata, in thetime-series data lake as part of the backup data. For example, the datalake service 110 may store the first logical backup and itscorresponding metadata (e.g., generated by metadata enrichment module114) in data lake 120A maintained for the Org1.

At 710, in the illustrated embodiment, the data protection service 102receives, from a data warehousing system, a first query specifying aparticular data view of the backup data associated with the firstorganization. For example, as described above with reference to FIG. 6,data warehouse 602 may issue a query 604 containing one or more searchcriteria to the data protection service 102. As a non-limiting example,the first query may be specified using the ODBC protocol. At 712, in theillustrated embodiment, the data protection service 102 retrieves theparticular data view from the time-series data lake based on the firstquery. For example, the data retrieval module 116 may use one or more ofthe criteria specified in the query 604 to parse the metadata in thedata lake 120A to retrieve the selected view 606 of the Org1's backupdata. At 714, in the illustrated embodiment, the data protection service102 provides the particular data view to the data warehousing system.For example, the data lake service 110 may then send the selected view606 of the Org1's data, from the data lake 120A, back to the datawarehouse 602.

As noted above, the particular data view provided by the data protectionservice 102 to the data warehouse 602 will vary depending on the query604 provided. For example, in some embodiments the particular data viewmay include a subset of the data from the first logical backup stored at708. Further, in some embodiments, the particular data view may includedata from two or more of the data sources associated with the Org1(e.g., at a particular point in time specified by the first query, froma particular time period specified in the first query, etc.). In someembodiments, the particular data view may include data from each of thedata stores 144 that the Org1 backs up using the data protection service102, though, in other embodiments, the particular data view may includedata from any subset of these data stores 144. In some embodiments, thequery 604 may specify a particular data store 144 and the particulardata view may include data from one or more backups of that particulardata store 144 (e.g., over a particular time period, at a particularpoint in time, etc.).

Example Embodiments: Providing Data Views from a Time-Series Data Laketo a Data Warehousing System

-   1. A method, comprising:

providing, by a cloud-based service, a data lake service that maintains,for a first organization, a time-series data lake storing a time-seriesrepresentation of backup data from a plurality of data sourcesassociated with the first organization;

receiving, by the cloud-based service, additional backup data associatedwith the first organization as part of one or more backup operations,wherein the additional backup data includes a first backup image, of afirst data source, that was generated at a first point in time;

creating, by the cloud-based service, a first logical backup of thefirst data source using the first backup image;

storing, by the cloud-based service, the first logical backup, andcorresponding metadata, in the time-series data lake as part of thebackup data;

receiving, by the cloud-based service from a data warehousing system, afirst query specifying a particular data view of the backup dataassociated with the first organization;

in response to the first query, retrieving, by the cloud-based service,the particular data view from the time-series data lake; and

providing, by the cloud-based service, the particular data view to thedata warehousing system.

-   2. The method of claim 1, wherein the particular data view includes    first data that was extracted from the first logical backup of the    first data source, and wherein the particular data view does not    include all of the data from the first logical backup of the first    data source.-   3. The method of claim 1, wherein the time-series representation of    the backup data includes metadata associated with the backup data,    wherein, for a given data element of the backup data in the    time-series data lake, the metadata indicates:

a particular one of the plurality of data sources from which the givendata element of the backup data originated; and

a time at which the given data element of the backup data was backed up;and

wherein the retrieving the particular data view includes parsing themetadata associated with the backup data based on one or more searchcriteria included in the first query.

-   4. The method of claim 1, wherein the plurality of data sources    includes a subset of two or more data sources, wherein the    particular data view includes data from each of the subset of two or    more data sources associated with the first organization.-   5. The method of claim 1, wherein the plurality of data sources    includes a subset of two or more data sources, wherein the    particular data view includes data from each of the subset of two or    more data sources from a particular time period specified in the    first query.-   6. The method of claim 1, wherein the particular data view includes    data from each of the plurality of data sources from a particular    point in time specified in the first query.-   7. The method of claim 1, wherein the particular data view includes    an entirety of the backup data from a particular one of the    plurality of data sources.-   8. The method of claim 1, wherein the particular data view includes    data from a particular data source, of the plurality of data    sources, from a particular time period specified in the first query.-   9. The method of claim 1, wherein the particular data view includes    data from a particular data source, of the plurality of data    sources, from a particular point in time specified in the first    query.-   10. A non-transitory, computer-readable medium having program    instructions stored thereon that are executable by one or more    computer systems to perform operations comprising:

providing a cloud-based data lake service that maintains data for aplurality of organizations, wherein, for a first one of the plurality oforganizations, the cloud-based data lake service maintains a time-seriesdata lake that stores a time-series representation of backup data from aplurality of data sources associated with the first organization;

receiving, from a data warehousing system, a first query specifying aparticular data view of the backup data associated with the firstorganization;

retrieving the particular data view from the time-series data lake basedon the first query; and

providing the particular data view to the data warehousing system.

-   11. The non-transitory, computer-readable medium of claim 10,    wherein the particular data view includes first data that was    extracted from a particular backup of a first one of the plurality    of data sources, and wherein the particular data view does not    include all of the data from the particular backup of the first data    source.-   12. The non-transitory, computer-readable medium of claim 10,    wherein the time-series representation of the backup data includes    metadata associated with the backup data, wherein, for a given data    element of the backup data in the time-series data lake, the    metadata indicates:

a particular one of the plurality of data sources from which the givendata element of the backup data originated; and

a time at which the given data element of the backup data was backed up;and

wherein the retrieving the particular data view includes parsing themetadata associated with the backup data based on one or more searchcriteria included in the first query.

-   13. The non-transitory, computer-readable medium of claim 10,    wherein the particular data view includes a subset of data from a    particular data source, of the plurality of data sources, across an    entirety of the time-series.-   14. A method, comprising:

receiving, by a cloud-based service, backup data associated with anorganization during one or more backup operations, wherein the backupdata includes:

-   -   a first physical backup, corresponding to a first data source,        generated at a first point in time; and    -   a second physical backup, corresponding to a second data source,        generated at a second point in time, wherein the second data        source is isolated from the first data source;

converting, by the cloud-based service, the backup data from one or morephysical backup formats, associated with the first and second datasources, into logical backup data;

storing, by the cloud-based service, the logical backup data andcorresponding metadata in a time-series data lake associated with theorganization;

receiving, by the cloud-based service from a data warehousing system, arequest for a particular data view of the backup data associated withthe organization;

retrieving, by the cloud-based service, the particular data view fromthe time-series data lake; and

providing, by the cloud-based service, the particular data view to thedata warehousing system.

-   15. The method of claim 14, wherein the particular data view    includes a subset of data from the first data source from a    particular time period specified in the request.-   16. The method of claim 14, wherein the particular data view    includes a subset of data from the first data source at a particular    point in time specified in the request.-   17. The method of claim 14, wherein the particular data view    includes a union of a first subset of data, from the first data    source, and a second subset of data from the second data source.-   18. The method of claim 14, wherein the particular data view    includes a union of a first subset of data, from the first data    source from a particular point in time, and a second subset of data    from the second data source from the particular point in time.-   19. The method of claim 14, wherein the particular data view    includes a union of a first subset of data, from the first data    source during a particular time period, and a second subset of data    from the second data source during the particular time period.-   20. The method of claim 14, wherein the request includes a first    query specified, by the data warehousing system, using the Open    Database Connectivity (“ODBC”) protocol.

Data Preservation Using a Time-Series Data Lake

In various embodiments, the disclosed systems and methods may be used toperform long-term data retention for an organization utilizing atime-series data lake. Such embodiments may prove particularly useful,as one non-limiting example, to preserve data during a legal hold. A“legal hold” (also known as a “litigation hold”) is a process in whichan organization (e.g., in response to instructions from theorganization's legal team) preserves electronically stored information(“ESI”) and paper documents that may be relevant to a pending oranticipated legal matter. For ESI, the organization's data custodian maybe asked to preserve relevant data for an indeterminate amount of time,until he or she is notified by the legal team that the legal matter isresolved or settled.

Complying with a legal hold request can present significant technicalchallenges. For example, as discussed above, organizations often storetheir production data in various (potentially disparate) data storesand, in many instances, this data is maintained in data silos. In such aconfiguration, some of the organization's data is kept isolated from,and thus not accessible to, other parts of the organization, making itmore difficult for the organization to search its data stores toidentify data relevant to the legal hold. Further, in addition topreserving relevant data in its production systems (e.g., data stores144 of FIG. 1), an organization may have relevant data stored in backupsystems that also needs to be preserved. To accomplish this, traditionalbackup systems, which store data in the form of backup images of theorganization's data stores, typically use a “brute force” approach inwhich, to preserve relevant data contained in a backup image, thesesystems retain the entire backup image (potentially terabytes in size),even if only a small fraction of the data (e.g., megabytes in size) inthe backup image is relevant to the legal hold. Further, thesetraditional backup systems may have to retain many such backups fromseveral weeks, months, or years, depending on the scope and time-windowof relevant data to be retained. This often results in such traditionalbackup systems storing large amounts of data for an indeterminate amountof time (e.g., months or years), given the open-ended nature of a legalhold.

In various embodiments, however, the disclosed systems and methodsprovide a technical solution to these technical problems. For example,in various embodiments, the disclosed data protection service 102 may beused to preserve a selected subset of data maintained in a data lake 120for an organization. For example, in various embodiments, thecloud-based data protection service 102 is operable to perform datapreservation (e.g., to comply with a legal hold request) of data storedin an organization's data lake 120 while avoiding the various technicalshortcomings of traditional backup systems described above. Forinstance, unlike traditional backup systems in which a backup image ofan entire data source (e.g., a database) is stored in a proprietaryformat (e.g., an RDS snapshot), the data protection service 102, asdescribed above, transforms an organization's backup data into alogical, addressable format (e.g., Apache™ Parquet format) that isenriched with various metadata information, such as timestamp, accesscontrol, and data source information. When a data preservation request(e.g., a legal hold request) is then received, the data protectionservice 102 may be used to identify and store data from the data lake120 that matches certain time, access control, or source criteriaspecified by the data preservation request.

In FIG. 8, a block diagram 800 depicts an example embodiment of dataprotection service 102. In the depicted embodiment, the data protectionservice 102 includes a data preservation service 802 and data lakestorage system 118, which, in turn, includes data lakes 120 for variousorganizations. In FIG. 8, data lake storage system 118 includes datalake 120A used to store backup data for Org1's various data stores 144in a logical, addressable format that, as described in more detailabove, is enriched with various items of metadata to facilitate improvedaccess. In various embodiments, data preservation service 802 isoperable to perform data preservation operations so as to identify andpreserve subsets of data for an organization that are maintained in thatorganization's data lake 120. For example, as shown in FIG. 8, dataprotection service 102 may receive a data preservation request 820 thatincludes one or more search criteria 822 and an organization identifier824. In various embodiments, data preservation service 802 is operableto identify data relevant to the data preservation request 820 based onthe one or more search criteria 822 and the org ID 824. The datapreservation service 802 may then preserve such data in a securelocation for an indeterminate amount of time. For example, in someembodiments, the data preservation request 820 may be sent to the dataprotection service 102 in conjunction with a legal hold, for which asubset of an organization's backup data is to be preserved until aparticular legal matter is resolved. In such embodiments, datapreservation service 802 may use the one or more search criteria 822 toidentify data relevant to the legal hold and store a copy of that datain a separate location where it may be preserved until the legal holdhas been lifted. Note, however, that this embodiment is described merelyas one non-limiting example and, in other embodiments, data preservationservice 802 may be used to preserve selected subsets of anorganization's data in any suitable context, as desired.

In the depicted embodiment, data preservation service 802 includessearch orchestrator module 804, which, in various embodiments, isoperable to request resources to run various tasks associated withperforming data preservation operations. For example, in variousembodiments, the search orchestrator module 804 operates in conjunctionwith a resource management module (not separately shown, for clarity) inthe data protection service 102 to allocate resources to perform thevarious data preservation operations described herein. Additionally, invarious embodiments, search orchestrator module 804 is operable togenerate a tag value that may be used to uniquely identify the datapreservation request 820 and a data preservation store 810 in which datadeemed relevant to the data preservation request 820 will be stored.Search orchestrator module 804 may generate the tag value using any ofvarious suitable techniques. In some embodiments for example, searchorchestrator module 804 may generate an identifiable tag value based onrequester's name, keywords identifying the litigation such as docketnumber or jurisdiction, and a timestamp at which data preservationrequest 820 was received. Further, in various embodiments, searchorchestrator module 804 is operable to generate a unique key value usingcryptographic methods where the seed is based on a custodian'sidentification credentials and a requester's identification credentialsso that both the custodian and the requester need to sign off before thepreserved data can be released from the litigation hold. As onenon-limiting example, in some embodiments the search orchestrator module804 may use one or more methods from the Java™ KeyGenerator class togenerate one or more key values using any of various suitablecryptographic algorithms (e.g., AES, DES, HmacSHA256, etc.). In variousembodiments, the search orchestrator module 804 is operable to generatemultiple keys (e.g., two, three, five, etc.), one for each of multipledifferent users, and require multiple parties to demonstrate theiragreement to deleting data from a data preservation store 810 by eachproviding their respective key. In some embodiments, a key value may beused to perform various authentication operations when a user attemptsto access the data maintained in a data preservation store 810.

Data preservation service 802 further includes search module 806, which,in various embodiments, is operable to search a time-series data lake120 to identify data that is relevant to a data preservation request820. For example, in various embodiments, search module 806 may selectthe data lake 120 associated with the org ID 624. Further, based on theone or more search criteria 822, the search module 806 may construct andexecute one or more queries against the selected data lake 120 toidentify data relevant to the data preservation request 820. In variousembodiments, search module 806 is operable to copy this identified datato a separate data preservation store 810. Note that, in variousembodiments, the data relevant to a data preservation request 820 mayinclude a subset (potentially a small subset) of the backup of a givendata source 144 from a given point in time. Unlike traditional backupsystems that would be required to retain an entire backup image simplyto preserve a small subset of the data contained therein, the discloseddata preservation service 802 is instead able to preserve that data thatis actually relevant to the data preservation request 820, savingdata-storage resources and improving operation of the data protectionservice 102 as a whole.

Further note that, in various embodiments, a data preservation request820 may be indefinite in duration due, for example, to the open-endednature of legal holds. Accordingly, in various embodiments, datapreservation service 802 may preserve relevant data in a datapreservation store 810 until the data preservation service 802 receivesa request from an authorized user (or from multiple authorized users inthe form of multi-person sign-off) to release or delete the data fromthe data preservation store 810. Thus, in various embodiments, datapreservation service 802 may retain data in the data preservation store810 using a data retention policy that differs from the data retentionpolicy of the data lake 120. For example, in some embodiments, anorganization or the data protection service 102 may establish a dataretention policy that dictates the duration for which data is maintainedin a particular data lake 120. Though the data retention period for agiven data lake 120 may be quite long (e.g., five years, 10 years, 15years, etc.), it may still be unsuitable for use in a data preservationcontext. For example, a legal hold may be initiated years after relevantdata has been stored in the data lake 120, and legal hold may last foran extended time (e.g. five years). If the data protection service 102were to simply evict data from the data lake 120 based on itspre-existing data retention policy, it is possible that data relevant tothe legal hold may be inadvertently deleted or otherwise lost. Byidentifying and copying data deemed relevant to the data preservationrequest 820, however, data preservation service 802 is able to maintainrelevant data in a secure data preservation store 810 for an open endedperiod of time without risk of the relevant data being deleted accordingto the data retention policy of the data lake 120.

Data preservation service 802 further includes data preservationmanagement module 808. In various embodiments, data preservationmanagement module 808 is operable to create a dedicated datapreservation store 810 for a given data preservation request 820. Insome embodiments, for example, data preservation store 810 may be adedicated sub-repository within the data lake 120 of the organizationfor which the data preservation operations are being performed. Note,however, that although the data preservation store 810 is part of theorganization's data lake 120 in the depicted embodiment, this embodimentis provided merely as one non-limiting example. In other embodiments,data preservation service 802 may instead (or additionally) store datadeemed relevant to the data preservation request 820 in any suitableformat using any suitable (local or remote) data storage device.Additionally, in some embodiments, the data preservation store 810 maybe hosted by a partnering application's data store that is purpose-buildfor specific applications (such as eDiscovery tools) and is fedon-demand by the data preservation service 802 using the organization'stime-series data lake 120. Further, in various embodiments, datapreservation management module 808 is operable to monitor new data as itis stored in the time-series data lake 120 (e.g., due to newer backupsbeing performed) and, if the new data contains records that match theone or more search criteria 822, the data preservation service 802 isoperable to automatically copy those data records into the datapreservation store 810 that matches the tag-value and key-valueassociated with the data preservation request 820.

Referring now to FIG. 9, a flow diagram illustrating an example method900 for preserving data from a time-series data lake is depicted,according to some embodiments. In various embodiments, method 900 may beperformed by data preservation service 802 of FIG. 8 to preserve asubset of data for an organization (such as Org1) for an indeterminateamount of time to comply with a data preservation request (e.g., a legalhold). As noted above, in various embodiments, data protection service102 (and, thus, data preservation service 802) may be implemented as acloud-based service using public or private cloud-based computingresources, such as server computer systems. In some such embodiments,the server computer system(s) used to implement the data preservationservice 802 may include (or have access to) a non-transitory,computer-readable medium having program instructions stored thereon thatare executable by the server computer system(s) to cause the operationsdescribed with reference to FIG. 9. In FIG. 9, method 900 includeselements 902-914. While these elements are shown in a particular orderfor ease of understanding, other orders may be used. In variousembodiments, some of the method elements may be performed concurrently,in a different order than shown, or may be omitted. Additional methodelements may also be performed as desired.

At 902, in the illustrated embodiment, the data preservation service 802receives a data preservation request 820. As discussed above, in someembodiments, the data preservation request 820 may be issued inconjunction with a legal hold in which a subset of an organization'sbackup data must be preserved for an indefinite period of time (e.g.,until the underlying legal matter has been resolved). In variousembodiments, the data preservation request 820 includes various searchcriteria, such as a relevant time period, keywords, user identifiers,data source identifiers, access control information, geographicalinformation, or any other criteria suitable to identify data relevant tothe data preservation request.

At 904, in the illustrated embodiment, the data preservation service 802generates a tag value and key-value associated with the datapreservation request. For example, as described above, in someembodiments the search orchestrator module 804 is operable to generate atag value that may be used to uniquely identify the data preservationrequest 820 and the data preservation store 810 in which data matchingthe search criteria 822 included in the data preservation request 820 isto be preserved. Additionally, in various embodiments, the searchorchestrator module 804 is operable to generate the key value that maybe used to perform authentication operations prior to providing arequesting user with access to the data in the data preservation store810. For example, in some embodiments, when a user requests access tothe data contained in the data preservation store 810, data preservationservice 802 may first require that the requesting user provide both theunique tag value and the key value so that the data preservation service802 may both identify the requested data preservation store 810 andverify that the requesting user is authorized to access the datacontained therein.

At 906, and the illustrated embodiment, the data preservation service802 establishes a data preservation store 810 in which to store datathat are deemed to match the search criteria identified in the datapreservation request 820. In some embodiments, the search orchestratormodule 804 may pass information associated with the data preservationrequest 820, such as the tag value or an identifier associated with thedata preservation request 820, to the data preservation managementmodule 808, which may then set up a new data preservation store 810. Forexample, in various embodiments, once the data preservation service 802receives the data preservation request 820, the data preservationmanagement module 808 may allocate a new data preservation store 810within the data lake 120 of the relevant organization (e.g., Org1, inthe current example). In some such embodiments, the data preservationstore 810 is a dedicated sub-repository that may be used to store a copyof the data deemed relevant to the data preservation request 820.

At 908, in the illustrated embodiment, the data preservation service 802searches the time-series data lake 120 for data matching the searchcriteria included in the data preservation request. For example, invarious embodiments, search module 806 is operable to parse theorganization's entire time-series data lake 120 to search for datarecords that are relevant to the data preservation request 820 (e.g., byquerying the time-series data lake 120A based on one or more searchcriteria 822 specified in the data preservation request 820). At 910, inthe illustrated embodiment, the data preservation service 802 stores acopy of the matching data records in the data preservation store 810. Asnon-limiting examples, note that, in various embodiments, the matchingdata records may include a subset of data from a backup of a particulardata source, all of the data from a backup of a particular data source,data from multiple different data sources, data from backups performedat different time periods from a single data source or multipledifferent data sources (e.g., data from a backup of data source 144Aperformed at time t₁ and data from a backup of data source 144Bperformed at time t₂).

At 912, in the illustrated embodiment, the data preservation service 802monitors incoming data for data records that match the search criteria822. For example, in various embodiments, the data preservationmanagement module 808 is operable to monitor incoming data streams intothe data lake 120. As new backup data is stored in the time-series datalake 120 (due to new backup operations), the data preservation service802 is operable, at element 914, to determine whether any of these newlyadded data records are relevant to the data preservation request 820. Ifso, method 900 proceeds to element 910, in which the records are copiedinto the data preservation store 810 associated with the datapreservation request 820. If, however, there are no newly added datarecords that are relevant to the data preservation request, method 900continues to element 912, in which it continues to monitor the incomingdata records. Note that, in some embodiments, the monitoring of element912 may be performed “inline” as data streams are arriving at the dataprotection service 102, the data lake 120, or the data preservationservice 802, for example through inline processing and filtering toharvest records matching search criteria. In other embodiments, however,the monitoring may be implemented as a post-process operation, forexample by batch processing and filtering at periodic intervals toharvest records matching search criteria.

As indicated in FIG. 9, elements 912-914 may be repeated (at any desiredfrequency, which may be specified by the organization, the dataprotection system, a data custodian, the issuer of the legal holdrequest, etc.) until the data preservation service 802 receives anotification that the data preservation request 820 has been lifted. Atsuch time, the data preservation service 802 may release (or activelydelete, in some embodiments) the data in the data preservation store810. In various embodiments, once the data preservation service 802 hasdeleted the data from the data preservation store 810, it may generate areport verifying the deletion and provide this report to one or moreusers (e.g., a data custodian for the organization, an issuer of thedata preservation request 820, etc.). Further note that, in variousembodiments, data preservation service 802 is operable to managemultiple different data-preservation jobs for a given organization(e.g., Org1) and for multiple different organizations at a given time.For example, in embodiments in which data protection service 102 storesdata associated with a first organization (Org1), a second organization(Org2), and a third organization (Org3), the data preservation service802 may perform one or more (simultaneous or otherwise overlapping)data-preservation jobs for one or more of Org1, Org2, and Org3.

Example Embodiments: Data Preservation Using a Time-Series Data Lake

-   1. A method, comprising:

maintaining, by a cloud-based service, a time-series data lake thatincludes, for an organization, a time-series representation of aplurality of data sources, wherein the time-series data lake retainsdata according to a first retention policy;

receiving, by the cloud-based service, a request for a subset of dataassociated with the organization;

retrieving, by the cloud-based service, the subset of data from thetime-series data lake; and

storing, by the cloud-based service, the subset of data in a particularstorage location that retains data according to a second, differentretention policy.

-   2. The method of claim 1, wherein the subset of data includes first    data from a backup of a first one of the plurality of data sources,    wherein the subset of data does not include all of the data from the    backup of the first data source.-   3. The method of claim 1, wherein the retrieving includes:

querying, by the cloud-based service, the time-series data lake based onone or more search criteria specified in the request for the subset ofdata.

-   4. The method of claim 1, wherein the particular storage location is    a dedicated sub-repository within the time-series data lake.-   5. The method of claim 1, further comprising:

maintaining, by the cloud-based service, backup data for theorganization, wherein the time-series representation of the plurality ofdata sources is generated based on the backup data for the organization.

-   6. The method of claim 1, wherein the request for the subset of data    includes one or more search criteria.-   7. The method of claim 6, wherein the one or more search criteria    include at least one of a data source identifier, a time identifier,    or an access control identifier.-   8. The method of claim 1, further comprising:

subsequent to the storing the subset of data in the particular storagelocation, receiving, by the cloud-based service, updated backup data forthe organization; and

storing, by the cloud-based service, the updated backup data in thetime-series data lake.

-   9. The method of claim 1, further comprising:

subsequent to the storing the subset of data in the particular storagelocation, monitoring, by the cloud-based service, updated backup dataadded to the time-series data lake to determine whether any of theupdated backup data matches one or more search criteria specified in therequest for the subset of data.

-   10. A non-transitory, computer-readable medium having program    instructions stored thereon that are executable by one or more    computer systems to perform operations comprising:

accessing a time-series data lake that includes, for an organization, atime-series representation of a plurality of data sources, wherein thetime-series data lake retains data according to a first retentionpolicy;

receiving a request for a subset of data associated with theorganization;

retrieving the subset of data from the time-series data lake; and

storing the subset of data in a particular storage location that retainsdata according to a second, different retention policy.

-   11. The non-transitory, computer-readable medium of claim 10,    wherein the retrieving the subset of data includes executing a query    against the time-series data lake at a first point in time, wherein    the subset of data includes data that matches the query at the first    point in time.-   12. The non-transitory, computer-readable medium of claim 10,    wherein the operations further comprise:

subsequent to the storing the subset of data in the particular storagelocation, receiving updated backup data for the organization;

storing the updated backup data in the time-series data lake; and

monitoring the updated backup data in the time-series data lake todetermine whether any of the updated backup data matches one or moresearch criteria included in the request for the subset of data.

-   13. The non-transitory, computer-readable medium of claim 12,    wherein the operations further comprise:

receiving information identifying a frequency with which the time-seriesdata lake is to be monitored for data matching the one or more searchcriteria, wherein the monitoring is performed periodically based on theidentified frequency.

-   14. The non-transitory, computer-readable medium of claim 12,    wherein the monitoring includes querying the time-series data lake    for data that was added to the time-series data lake subsequent to a    first point in time at which the subset of data was retrieved from    the time-series data lake.-   15. The non-transitory, computer-readable medium of claim 12,    wherein the operations further comprise:

based on the monitoring, identifying a second subset of data associatedwith the organization that matches the one or more search criteria; and

storing the second subset of data in the particular storage location.

-   16. A method, comprising:

providing, by a cloud-based service, a data lake service that maintainsdata for a plurality of organizations, wherein, for a first one of theplurality of organizations, the data lake service maintains atime-series data lake that stores a time-series representation of dataassociated with the first organization according to a first retentionpolicy;

receiving, by the cloud-based service from a requesting entity, arequest to preserve backup data associated with the first organizationthat matches one or more search criteria; and

storing, by the cloud-based service, a subset of data that matches theone or more search criteria in a particular storage location, whereinthe particular storage location that retains data according to a second,different retention policy.

-   17. The method of claim 16, wherein, based on the second, different    retention policy, the cloud-based service retains data in the    particular storage location until a second request is received, from    one or more authorized users, to release the subset of data.-   18. The method of claim 16, further comprising:

encrypting, by the cloud-based service, the subset of data to generatean encrypted subset of data, wherein the storing includes storing theencrypted subset of data in the particular storage location.

-   19. The method of claim 16, further comprising:

generating, by the cloud-based service, for the subset of data, a tagvalue that is usable to identify the subset of data that matches the oneor more search criteria; and

wherein the storing the subset of data includes storing the tag value inthe particular storage location with the subset of data.

-   20. The method of claim 16, further comprising:

subsequent to the storing, receiving, by the cloud-based service from arequesting user, a subsequent request to access the subset of data;

initiating, by the cloud-based service, one or more authenticationoperations for the requesting user; and

in response to the requesting user satisfying the one or moreauthentication operations, providing, by the cloud-based service, thesubset of data to the requesting user.

Modification of Data in a Time-Series Data Lake

In various embodiments, in addition to performing data retentionoperations described above with reference to FIGS. 8-9, the disclosedsystems and methods may be used to modify specific data items within anorganization's backup data maintained in the data lake 120. For example,organizations are often required to protect growing volumes of personaland sensitive data and to adhere to various data privacy regulations,such as the General Data Protection Regulation (“GDPR”) and theCalifornia Consumer Privacy Act (“CCPA”). Adhering to such data privacyregulations can present various technical challenges. For example,complying with “Right to Be Forgotten” requests (in which an individual(e.g., in an EU member nation) requests that an organization modify(e.g., delete, mask, anonymized, etc.) some or all of the data about theindividual that the organization maintains) can be particularlydifficult.

For traditional image-based backup systems that store data in the formof backup images of an entire system or data source, deleting therelevant information—even if it is just a small fraction of the overallbackup image—is a time-intensive and computationally expensive task thatrequires deleting the entire backup image or copying the data to a newimage where the information “to be forgotten” may be deleted. Sincebackup images in these traditional backup systems are typicallypreserved for an extended period of time (e.g. years), complying with a“Right to Be Forgotten” request is a burdensome and resource-intensiveoperation even for a single request from a data subject (e.g., a user,group of users, customer, employee, or any other individual(s) orentities for which an organization may maintain data). Additionally, inmany instances, deleting an entire backup image is simply not possible.For example, an organization may need to retain the remainder of thedata in that backup image for business continuity and recovery purposes.In some instances, some types of data needs be preserved for regulatorycompliance. Further, in some instances, the regulations that provide the“Right to Be Forgotten” may have one or more exception clauses. As anon-limiting example, the CCPA provides that information does not needto be deleted if it is necessary to “protect against malicious,deceptive, fraudulent, or illegal activity.” Another non-limitingexample is a legal hold in which data must be preserved pending anon-going legal matter, even if a data subject whose data is subject tothe legal hold has requested deletion of his or her data.

In various embodiments, the disclosed systems and methods address thesetechnical shortcomings of prior image-based backup systems, therebyimproving the process of modifying (e.g., deleting) data maintained in abackup system. For example, in FIG. 10, block diagram 1000 depicts anexample embodiment of a data protection service 102, which includes adata modification service 1002 and data lake storage system 118. Invarious embodiments, data modification service 1002 is operable toidentify data associated with a requesting user (a data subject) that ismaintained in a data lake 120 in a precise or “surgical” manner andperform the requested modification on that user's data in a way thatavoids the various technical problems of traditional image-based backupsystems. For example, as shown in FIG. 10, data protection service 102may receive a data modification request 1020, which may include variousitems of information. In the depicted embodiment, the data modificationrequest 1020 includes a user ID 1022 identifying a data subjectrequesting his or her data to be modified, an organization ID 1024identifying the organization with which the data subject is associated,a scope parameter 1026 indicating the scope of the user's data to bemodified, and a requested modification identifier 1028 identifying themodification(s) the data subject wishes to make to his or her data. Insome embodiments, the scope parameter 1026 may be used to limit themodification criteria based on metadata such as timestamps, accesscontrol lists, tags associated with the objects, file system-levelpermissions, record-level security (if applicable), the nature orclassification of each column, object-level timestamps, file- orrecord-level checksum or hashed fingerprints, etc. Note, however, thatthis embodiment is provided merely as one non-limiting example and, inother embodiments, additional or fewer items of information may beincluded in a given data modification request 1020. Further note that,in various embodiments, rather than submitting the data modificationrequest 1020 his or herself, the data subject may instead submit a“Right to Be Forgotten” request to an organization (e.g., Org1) thatmaintains data associated with the user. After receiving such a request,that organization may create and send data modification request 1020 tothe data protection service 102.

In the depicted embodiment, data modification service 1002 includessearch orchestrator module 1004, which, in various embodiments, isoperable to request resources to run various tasks associated withperforming data modification operations. For example, in variousembodiments, search orchestrator module 1004 operates in conjunctionwith a resource management module (not shown separately, for clarity) inthe data protection service 102 to allocate resources to perform thevarious data modification operations described herein. Additionally, invarious embodiments, search orchestrator module 1004 is operable togenerate a tag value that may be used to uniquely identify the datamodification request 1020. Further, in various embodiments, searchorchestrator module 1004 is operable to generate a unique key valueusing cryptographic methods where the seed is based on users'credentials or privileges, where the key value may be used performvarious authentication operations when a user attempts various actions,such as status checks, auditing, and reporting status of the datamodification request 1020. In various embodiments, search orchestratormodule 1004 may generate key values and tag values using any of varioussuitable techniques, including those described above with reference tosearch orchestrator module 804 of FIG. 8.

Data modification service 1002 further includes search module 1006,which, in various embodiments, is operable to search a time-series datalake 120 to identify data deemed relevant to the data modificationrequest 1020. For example, in various embodiments, search module 1006may select the data lake 120 associated with the org ID 1024, as that isthe data lake 120 that includes data associated with the data subject.In various embodiments, search module 1006 is operable to identify andfetch all data relevant to the data subject associated with the user ID1022. (Note that, in embodiments in which the data modification request1020 is provided for a group of users, the request 1020 may includemultiple user ID's 1022 associated with the various data subjectsincluded in the group.) For example, in various embodiments, searchmodule 1006 may parse the data in the data lake 120 to locate recordsmatching the criteria associated with the data modification request1020, such as the user ID 1022, time window for information, accesscontrol information, source information, or any other suitable searchparameters (which may be specified in the data modification request1020). Note that, in various embodiments, since the organization's(Org1, in the current example) backup data is stored in the data lake120 enriched with various items of metadata, the search module 1006 isoperable to quickly search the entire data lake 120A for data recordsthat match the data modification request 1020. In various embodiments,after identifying data relevant to the data modification request 1020,the data modification service 1002 may store the relevant data in a datapreservation store 810 (discussed above) for “staging,” so that one ormore users (such as the data subject or one or more users associatedwith the relevant organization) may review the data prior to performingthe requested modification.

Data modification service 1002 further includes data modificationmanagement module 1008. In various embodiments, once the relevant datahas been identified, data modification management module 1008 isoperable to generate a report that includes information indicative ofthe relevant data. As a non-limiting example, the report may include asample of the data records that match the “Right to Be Forgotten”request criteria, information indicating the data source 144 from whichthe data records originated, timestamp information associated with thedata records, or any other suitable items of information. In variousembodiments, such a report may be provided to one or more usersassociated with the organization (e.g., Org1) for approval prior toperforming the requested modification on the relevant data.

In various embodiments, if the requested modification is approved by theappropriate users of the organization, the data modification service1002 may perform the requested modification identified by the requestedmodification identifier 1028 of the data modification request 1020. Insome embodiments, the requested modification may include erasing alldata records matching the “Right to Be Forgotten” criteria, which wouldresult in a deletion of the data records from the data lake 120. Inother embodiments, the requested action may include masking some or allof the data records matching the “Right to Be Forgotten” criteria. Insome such embodiments, this process includes masking data values in therelevant records with predefined default values while maintaining thestructure of the data records themselves. As a non-limiting example,masking data record may include replacing one or more items of the datasubject's personally identifiable information (e.g., name, residenceaddress, phone number, etc.) with predetermined default values (e.g.,“John Doe,” [null], 1-111-1111, etc.). Further, in some embodiments, therequested modification may include anonymizing all data records matchingthe “Right to Be Forgotten” criteria. In some such embodiments, thisprocess masks just the personally identifiable information related tothe data subject. In various embodiments, the disclosed datamodification service 1002 is capable of performing these modificationsin a “surgical” and precise manner due, in part, to the way in which thedata lake service 110 converts an organization's backup data from abackup image to a logical backup and enriches the logical data withvarious items of metadata before storing it in a data lake 120, asdescribed above with reference to FIGS. 1-4.

In various embodiments, the data modification service 1002 may then passthe requested action to a policy manager (not separately shown, forclarity) in the data protection service 102 to secure resources from aresource management module, which may register and queue the request forexecution. The data modification service 1002 may then return the tagvalue and key value associated with the data modification request 1020to one or more users associated with the organization (e.g., a datacustodian). In various embodiments, this tag value and key value may beused for status checks, auditing, and reporting.

In some embodiments, prior to performing the requested modification, thedata modification service 1002 may contact the data preservation service802 (described above with reference to FIG. 8) to determine whetherthere are records deemed relevant to the data modification request 1020that are also subject to an on-going data preservation request 820. Ifso, in various embodiments, those records are identified and excludedfrom the requested data modification so as to avoid deleting ormodifying data that is otherwise subject to a data preservation request.In some such embodiments, the determination of whether to modify orpreserve a data subject's data may be delegated to a user (e.g., theorganization's data custodian) in the event of a conflict between a datapreservation request 820 and a data modification request 1020. Notethat, in some embodiments, the data preservation requests 820 and thedata modification requests 1020 may both be defined by one or moresearch criteria or one or more scope parameters, or both. In variousembodiments, if the data modification service 1002 or data preservationservice 802 detects an overlap in search criteria or scope parameter(s)between a data preservation request 820 and a data modification request1020, the preservation operation may take precedence of thedata-modification request in accordance with applicable laws orregulations (e.g., California's CCPA). As noted above, the applicabledata privacy regulations may vary depending on the jurisdiction in whichan organization resides or operates. Accordingly, in some embodiments,the disclosed techniques include utilizing a policy engine to resolveconflicts between data preservation requests 820 and data modificationrequests 1020 based on the relevant organization's jurisdiction andapplicable data privacy regulations, for example by comparing therequirements of the applicable data privacy regulation(s) to theconflicting data preservation request 820 to determine whether one ormore items of data must be preserved (per the data preservation request820) despite the receipt of a data modification request 1020.

In various embodiments, the data modification service 1002 may thenperform the requested data-modification operation on the matching datarecords using the resources allocated by the resource management module.Once the requested action has been performed, the data modificationservice 1002 may then provide notification to one or more users, such asthe data custodian of the relevant organization or the data subject.

Note that, in various embodiments, the data modification service 1002 isoperable to continue monitoring incoming data streams into the data lake120. As new data is stored in the data lake 120 (due to newer backups),the data modification service 1002 may determine whether any of thesenewly added records are relevant to any previously issued datamodification requests 1020. If so, the data modification service 1002may automatically flag these data records (and, optionally, store themin a data preservation store 810) so that the requested modification maybe performed on these data records as well. Note that, in someembodiments, the data modification service 1002 is operable to performthis monitoring “inline” as data streams are arriving at the dataprotection service 102, the data lake 120, or the data modificationservice 1002, for example through inline processing and filtering toharvest records matching search criteria. In other embodiments, however,the data modification service 1002 may implement the monitoring as apost-process operation, for example by batch processing and filtering atperiodic intervals to harvest records matching search criteria.

Referring now to FIG. 11, a flow diagram illustrating an example method1100 for modifying data in a time-series data lake is depicted,according to some embodiments. In various embodiments, method 1100 maybe performed by data modification service 1002 of FIG. 10 to modify(e.g., delete, mask, anonymize, etc.) data associated with a datasubject (e.g., a user of the service provided by an organization) Asnoted above, in various embodiments, data protection service 102 (and,thus, data modification service 1002) is implemented as a cloud-basedservice using public or private cloud-based computing resources such asserver computer systems. In some such embodiments, the server computersystem(s) used to implement the data modification service 1002 mayinclude (or have access to) a non-transitory, computer-readable mediumhaving program instructions stored thereon that are executable by theserver computer system(s) to cause the operations described withreference to FIG. 11. In FIG. 11, method 1100 includes elements1102-1108. While these elements are shown in a particular order for easeof understanding, other orders may be used. In various embodiments, someof the method elements may be performed concurrently, in a differentorder than shown, or may be omitted. Additional method elements may alsobe performed as desired.

At 1102, in the illustrated embodiment, the data protection service 102maintains a data lake 120 that includes, for the first organization, atime-series representation of data from a plurality of data sources 144associated with the organization. At 1104, in the illustratedembodiment, the data protection service 102 receives a request 1020(e.g., corresponding to a “Right to Be Forgotten” request) to modifydata associated with a first user of the first organization. In variousembodiments, the request 1020 may include one or more items ofinformation, such as one or more search parameters or an indication ofone or more modifications to be performed. For example, in theembodiment of FIG. 10, the data modification request 1020 includes auser ID 1022, an org ID 1024, scope parameters 1026, and a requestedmodification identifier 1028. Note, however, that this embodiment isprovided nearly as one non-limiting example.

At 1106, in the illustrated embodiment, the data modification service1002 searches the data lake 120 to identify a subset of data associatedwith the first user that matches the one or more search parameters. Forexample, since, in various embodiments, the backup data stored in datalake 120 is enriched with one or more items of metadata, the searchmodule 1006 may use the criteria provided in the request 1020 to parsethe metadata in the data lake 120, enabling the search module 1006 toefficiently identify the subset of data that is associated with thefirst user and that matches the one or more search parameters. At 1108,in the illustrated embodiment, the data modification service 1002performs one or more modifications on the subset of data that matchesthe one or more search parameters. As noted above, performing these oneor more modifications may include deleting some or all of the subset ofdata, masking some or all of the subset of data, anonymizing some or allof the subset of data, etc. For example, in instances in which thesubset of data includes one or more items of personal information (e.g.,name, residence, contact information, SSN, email address, etc.), element1108 may include modifying one or more items of personal information,for example by replacing at least one item of the personal informationwith a corresponding default value. Further note that, in variousembodiments, the data modification service 1002 may perform differentmodifications on different portions of the subset of data. For example,in some embodiments, the data modification service 1002 may delete alldata records associated with the user that include personallyidentifiable information (such as the data subject's name, SocialSecurity number, etc.) while simply masking or anonymizing other datarecords associated with the user.

Note that, in some embodiments, the data modification service 1002 mayperform various operations prior to performing the one or moremodifications at 1108. For example, in some embodiments, prior toperforming the one or more modifications, the data modification service1002 may store the subset of data in a particular storage location(e.g., data preservation store 810) that retains data according to asecond retention policy that is different from a first retention policyused by the time-series data lake 120 for the organization. In some suchembodiments, the data modification service 1002 may then provide theuser (e.g., the requesting user) with access to the subset of data inthe particular storage area for the user's review. Further, in someembodiments, prior to performing the one or more modifications, the datamodification service 1002 may verify that the subset of data is notsubject to a data preservation request associated with the organization.Additionally, in some embodiments, prior to performing the one or moremodifications, the data modification service 1002 may generate a reportcorresponding to the subset of data, where the report may includevarious items of information about the subset of data, such as the datasources from which the subset of data was retrieved, timelineinformation, sample data records, etc. The data modification service1002 may then provide this report to one or more users for approvalprior to performing the one or more operations on the identified subsetof data.

Further, in some embodiments, subsequent to performing the one or moremodifications on the subset of data, the data modification service 1002may monitor updated backup data associated with the organization (e.g.,before, as, or after that backup data is stored in the time-series datalake 120) to determine whether any of the updated backup data matchesthe one or more search parameters included in the data modificationrequest 1020. If any of the updated backup data does match the datamodification request 1020, the data modification service 1002 mayautomatically flag these data records (and, optionally, store them in adata preservation store 810) so that the requested modification(s) mayalso be performed on this data.

Example Embodiments: Modification of Data in a Time-Series Data Lake

-   1. A method, comprising:

maintaining, by a cloud-based service, a time-series data lake thatincludes, for an organization, a time-series representation of data froma plurality of data sources;

receiving, by the cloud-based service, a request to modify dataassociated with a user of the organization, wherein the requestincludes:

-   -   one or more search parameters; and    -   an indication of one or more modifications to be performed;

parsing, by the cloud-based service, the time-series data lake toidentify a subset of data that matches the one or more searchparameters; and

performing, by the cloud-based service, the one or more modifications onthe subset of data in the time-series data lake.

-   2. The method of claim 1, wherein the performing the one or more    modifications includes deleting at least a portion of the subset of    data from the time-series data lake.-   3. The method of claim 1, wherein the subset of data associated with    the user includes one or more items of personal information, and    wherein the performing the one or more modifications includes    modifying the one or more items of personal information within the    subset of data.-   4. The method of claim 3, wherein the one or more items of personal    information includes at least one of a name of the user, a residence    of the user, and contact information associated with the user.-   5. The method of claim 3, wherein the modifying the one or more    items of personal information includes replacing the one or more    items of personal information with one or more corresponding default    values.-   6. The method of claim 1, wherein the performing the one or more    modifications includes masking one or more values of data records    included in the subset of data.-   7. The method of claim 1, wherein the time-series data lake retains    data according to a first retention policy, the method further    comprising:

prior to the performing the one or more modifications, storing, by thecloud-based service, the subset of data in a particular storage locationthat retains data according to a second, different retention policy; and

providing, by the cloud-based service, the user with access to thesubset of data in the particular storage location for review.

-   8. The method of claim 1, further comprising:

prior to the performing the one or more modifications, verifying, by thecloud-based service, that the subset of data is not subject to a datapreservation request associated with the organization.

-   9. The method of claim 1, further comprising:

subsequent to the performing the one or more modifications on the subsetof data, receiving, by the cloud-based service, updated backup data forthe organization;

storing, by the cloud-based service, the updated backup data in thetime-series data lake; and

monitoring, by the cloud-based service, the updated backup data in thetime-series data lake to determine whether any of the updated backupdata matches the one or more search parameters.

-   10. The method of claim 1, further comprising:

prior to the performing the one or more modifications, generating, bythe cloud-based service, a report that includes informationcorresponding to the subset of data that matches the one or more searchparameters; and

providing, by the cloud-based service, the report to one or more usersfor approval prior to the performing the one or more modifications.

-   11. A non-transitory, computer-readable medium having program    instructions stored thereon that are executable by a computer system    to perform operations comprising:

accessing a time-series data lake that includes, for an organization, atime-series representation of data from a plurality of data sources;

receiving a request to modify data associated with a user of theorganization, wherein the request includes:

-   -   one or more search parameters; and    -   an indication of one or more modifications to be performed;

parsing the time-series data lake to identify a subset of data thatmatches the one or more search parameters; and

performing the one or more modifications on the subset of data in thetime-series data lake.

-   12. The non-transitory, computer-readable medium of claim 11,    wherein the performing the one or more modifications includes    deleting at least a portion of the subset of data from the    time-series data lake.-   13. The non-transitory, computer-readable medium of claim 11,    wherein the performing the one or more modifications includes    masking one or more values of data records included in the subset of    data.-   14. The non-transitory, computer-readable medium of claim 11,    wherein the time-series data lake retains data according to a first    retention policy, wherein the operations further comprise:

prior to the performing the one or more modifications, storing thesubset of data in a particular storage location that retains dataaccording to a second, different retention policy; and

providing the user with access to the subset of data in the particularstorage location for review.

-   15. The non-transitory, computer-readable medium of claim 11,    wherein the operations further comprise:

prior to the performing the one or more modifications, verifying thatthe subset of data is not subject to a data preservation requestassociated with the organization.

-   16. A system, comprising:

at least one processor;

a non-transitory, computer-readable medium having instructions storedthereon that are executable by the at least one processor to cause thesystem to:

-   -   access a time-series data lake that includes, for an        organization, a time-series representation of data from a        plurality of data sources;    -   receive a request to modify data associated with a user of the        organization, wherein the request includes:        -   one or more search parameters; and        -   an indication of one or more modifications to be performed;    -   parse the time-series data lake to identify a subset of data        that matches the one or more search parameters; and    -   perform the one or more modifications on the subset of data in        the time-series data lake.

-   17. The system of claim 16, wherein the subset of data associated    with the user includes one or more items of personal information,    and wherein, to perform the one or more modifications, the    instructions are further executable by the at least one processor to    cause the system to modify the one or more items of personal    information within the subset of data.

-   18. The system of claim 17, wherein, to perform the one or more    modifications, the instructions are further executable by the at    least one processor to cause the system to replace the one or more    items of personal information with one or more corresponding default    values.

-   19. The system of claim 16, wherein the instructions are further    executable by the at least one processor to cause the system to:

in response to receiving updated backup data for the organization, storethe updated backup data in the time-series data lake; and

monitor the updated backup data in the time-series data lake todetermine whether any of the updated backup data matches the one or moresearch parameters.

-   20. The system of claim 16, wherein the instructions are further    executable by the at least one processor to cause the system to:

prior to performing the one or more modifications, generate a reportthat includes information corresponding to the subset of data thatmatches the one or more search parameters; and

provide the report to one or more users for approval prior to theperforming the one or more modifications.

Example Computer System

Referring now to FIG. 12, a block diagram of an example computer system1200 is depicted, which may implement one or more computer systems, suchas one or more cloud-based server computer systems used to implement thedata protection service 102 of FIG. 1, according to various embodiments.Computer system 1200 includes a processor subsystem 1220 that is coupledto a system memory 1240 and I/O interfaces(s) 1260 via an interconnect1280 (e.g., a system bus). I/O interface(s) 1260 is coupled to one ormore I/O devices 1270. Computer system 1200 may be any of various typesof devices, including, but not limited to, a server computer system,personal computer system, desktop computer, laptop or notebook computer,mainframe computer system, server computer system operating in adatacenter facility, workstation, network computer, etc. Although asingle computer system 1200 is shown in FIG. 12 for convenience,computer system 1200 may also be implemented as two or more computersystems operating together.

Processor subsystem 1220 may include one or more processors orprocessing units. In various embodiments of computer system 1200,multiple instances of processor subsystem 1220 may be coupled tointerconnect 1280. In various embodiments, processor subsystem 1220 (oreach processor unit within 1220) may contain a cache or other form ofon-board memory.

System memory 1240 is usable to store program instructions executable byprocessor subsystem 1220 to cause system 1200 perform various operationsdescribed herein. System memory 1240 may be implemented using differentphysical, non-transitory memory media, such as hard disk storage, floppydisk storage, removable disk storage, flash memory, random access memory(RAM-SRAM, EDO RAM, SDRAM, DDR SDRAM, RAMBUS RAM, etc.), read onlymemory (PROM, EEPROM, etc.), and so on. Memory in computer system 1200is not limited to primary storage such as system memory 1240. Rather,computer system 1200 may also include other forms of storage such ascache memory in processor subsystem 1220 and secondary storage on I/Odevices 1270 (e.g., a hard drive, storage array, etc.). In someembodiments, these other forms of storage may also store programinstructions executable by processor subsystem 1220.

I/O interfaces 1260 may be any of various types of interfaces configuredto couple to and communicate with other devices, according to variousembodiments. In one embodiment, I/O interface 1260 is a bridge chip(e.g., Southbridge) from a front-side to one or more back-side buses.I/O interfaces 1260 may be coupled to one or more I/O devices 1270 viaone or more corresponding buses or other interfaces. Examples of I/Odevices 1270 include storage devices (hard drive, optical drive,removable flash drive, storage array, SAN, or their associatedcontroller), network interface devices (e.g., to a local or wide-areanetwork), or other devices (e.g., graphics, user interface devices,etc.). In one embodiment, I/O devices 1270 includes a network interfacedevice (e.g., configured to communicate over WiFi, Bluetooth, Ethernet,etc.), and computer system 1200 is coupled to a network via the networkinterface device.

The present disclosure includes references to “embodiments,” which arenon-limiting implementations of the disclosed concepts. References to“an embodiment,” “one embodiment,” “a particular embodiment,” “someembodiments,” “various embodiments,” and the like do not necessarilyrefer to the same embodiment. A large number of possible embodiments arecontemplated, including specific embodiments described in detail, aswell as modifications or alternatives that fall within the spirit orscope of the disclosure. Not all embodiments will necessarily manifestany or all of the potential advantages described herein.

Unless stated otherwise, the specific embodiments described herein arenot intended to limit the scope of claims that are drafted based on thisdisclosure to the disclosed forms, even where only a single example isdescribed with respect to a particular feature. The disclosedembodiments are thus intended to be illustrative rather thanrestrictive, absent any statements to the contrary. The application isintended to cover such alternatives, modifications, and equivalents thatwould be apparent to a person skilled in the art having the benefit ofthis disclosure.

Particular features, structures, or characteristics may be combined inany suitable manner consistent with this disclosure. The disclosure isthus intended to include any feature or combination of featuresdisclosed herein (either explicitly or implicitly), or anygeneralization thereof. Accordingly, new claims may be formulated duringprosecution of this application (or an application claiming prioritythereto) to any such combination of features. In particular, withreference to the appended claims, features from dependent claims may becombined with those of the independent claims and features fromrespective independent claims may be combined in any appropriate mannerand not merely in the specific combinations enumerated in the appendedclaims.

For example, while the appended dependent claims are drafted such thateach depends on a single other claim, additional dependencies are alsocontemplated, including the following: Claim 3 (could depend from any ofclaims 1-2); claim 4 (any preceding claim); claim 5 (claim 4), etc.Where appropriate, it is also contemplated that claims drafted in onestatutory type (e.g., apparatus) suggest corresponding claims of anotherstatutory type (e.g., method).

Because this disclosure is a legal document, various terms and phrasesmay be subject to administrative and judicial interpretation. Publicnotice is hereby given that the following paragraphs, as well asdefinitions provided throughout the disclosure, are to be used indetermining how to interpret claims that are drafted based on thisdisclosure.

References to the singular forms such “a,” “an,” and “the” are intendedto mean “one or more” unless the context clearly dictates otherwise.Reference to “an item” in a claim thus does not preclude additionalinstances of the item.

The word “may” is used herein in a permissive sense (i.e., having thepotential to, being able to) and not in a mandatory sense (i.e., must).

The terms “comprising” and “including,” and forms thereof, areopen-ended and mean “including, but not limited to.”

When the term “or” is used in this disclosure with respect to a list ofoptions, it will generally be understood to be used in the inclusivesense unless the context provides otherwise. Thus, a recitation of “x ory” is equivalent to “x or y, or both,” covering x but not y, y but notx, and both x and y. On the other hand, a phrase such as “either x or y,but not both” makes clear that “or” is being used in the exclusivesense.

A recitation of “w, x, y, or z, or any combination thereof” or “at leastone of . . . w, x, y, and z” is intended to cover all possibilitiesinvolving a single element up to the total number of elements in theset. For example, given the set [w, x, y, z], these phrasings cover anysingle element of the set (e.g., w but not x, y, or z), any two elements(e.g., w and x, but not y or z), any three elements (e.g., w, x, and y,but not z), and all four elements. The phrase “at least one of . . . w,x, y, and z” thus refers to at least one of element of the set [w, x, y,z], thereby covering all possible combinations in this list of options.This phrase is not to be interpreted to require that there is at leastone instance of w, at least one instance of x, at least one instance ofy, and at least one instance of z.

Various “labels” may proceed nouns in this disclosure. Unless contextprovides otherwise, different labels used for a feature (e.g., “firstcircuit,” “second circuit,” “particular circuit,” “given circuit,” etc.)refer to different instances of the feature. The labels “first,”“second,” and “third” when applied to a particular feature do not implyany type of ordering (e.g., spatial, temporal, logical, etc.), unlessstated otherwise.

Within this disclosure, different entities (which may variously bereferred to as “units,” “circuits,” other components, etc.) may bedescribed or claimed as “configured” to perform one or more tasks oroperations. This formulation—“[entity] configured to [perform one ormore tasks]”—is used herein to refer to structure (i.e., somethingphysical). More specifically, this formulation is used to indicate thatthis structure is arranged to perform the one or more tasks duringoperation. A structure can be said to be “configured to” perform sometask even if the structure is not currently being operated. A “memorydevice configured to store data” is intended to cover, for example, anintegrated circuit that has circuitry that performs this function duringoperation, even if the integrated circuit in question is not currentlybeing used (e.g., a power supply is not connected to it). Thus, anentity described or recited as “configured to” perform some task refersto something physical, such as a device, circuit, memory storing programinstructions executable to implement the task, etc. This phrase is notused herein to refer to something intangible.

The term “configured to” is not intended to mean “configurable to.” Anunprogrammed FPGA, for example, would not be considered to be“configured to” perform some specific function. This unprogrammed FPGAmay be “configurable to” perform that function, however.

Reciting in the appended claims that a structure is “configured to”perform one or more tasks is expressly intended not to invoke 35 U.S.C.§ 112(f) for that claim element. Should Applicant wish to invoke Section112(f) during prosecution, it will recite claim elements using the“means for [performing a function]” construct.

The phrase “based on” is used to describe one or more factors thataffect a determination. This term does not foreclose the possibilitythat additional factors may affect the determination. That is, adetermination may be solely based on specified factors or based on thespecified factors as well as other, unspecified factors. Consider thephrase “determine A based on B.” This phrase specifies that B is afactor that is used to determine A or that affects the determination ofA. This phrase does not foreclose that the determination of A may alsobe based on some other factor, such as C. This phrase is also intendedto cover an embodiment in which A is determined based solely on B. Asused herein, the phrase “based on” is synonymous with the phrase “basedat least in part on.”

The phrase “in response to” describes one or more factors that triggeran effect. This phrase does not foreclose the possibility thatadditional factors may affect or otherwise trigger the effect. That is,an effect may be solely in response to those factors, or may be inresponse to the specified factors as well as other, unspecified factors.Consider the phrase “perform A in response to B.” This phrase specifiesthat B is a factor that triggers the performance of A. This phrase doesnot foreclose that performing A may also be in response to some otherfactor, such as C. This phrase is also intended to cover an embodimentin which A is performed solely in response to B.

In this disclosure, various “modules” operable to perform designatedfunctions are shown in the figures and described in detail (e.g.,conversion module 112, metadata enrichment module 114, data retrievalmodule 116, etc.). As used herein, a “module” refers to software orhardware that is operable to perform a specified set of operations. Amodule may refer to a set of software instructions that are executableby a computer system to perform the set of operations. A module may alsorefer to hardware that is configured to perform the set of operations. Ahardware module may constitute general-purpose hardware as well as anon-transitory computer-readable medium that stores programinstructions, or specialized hardware such as a customized ASIC.Accordingly, a module that is described as being “executable” to performoperations refers to a software module, while a module that is describedas being “configured” to perform operations refers to a hardware module.A module that is described as “operable” to perform operations refers toa software module, a hardware module, or some combination thereof.Further, for any discussion herein that refers to a module that is“executable” to perform certain operations, it is to be understood thatthose operations may be implemented, in other embodiments, by a hardwaremodule “configured” to perform the operations, and vice versa.

What is claimed is:
 1. A method, comprising: providing, by a cloud-basedservice, a data lake service that maintains data for a plurality oforganizations, wherein, for a first one of the plurality oforganizations, the data lake service maintains a time-series data lakethat stores a time-series representation of data associated with thefirst organization; receiving, by the cloud-based service, backup data,associated with the first organization, from a plurality of datasources; generating, by the cloud-based service, metadata associatedwith the backup data, wherein, for a given data element of the backupdata, corresponding metadata includes source information identifying aparticular one of the plurality of data sources from which the givendata element of the backup data originated; and storing, by thecloud-based service, the backup data and the metadata in the time-seriesdata lake.
 2. The method of claim 1, wherein the backup data includes: afirst physical backup, created at a first point in time, of a first oneof the plurality of data sources, wherein the first physical backup isprovided in a first format; and a second physical backup, created at asecond point in time, of a second one of the plurality of data sources,wherein the second physical backup is provided in a second format. 3.The method of claim 2, wherein the first physical backup is one of thefollowing types of physical backups: an Amazon™ RDS snapshot; an Amazon™DynamoDB database snapshot; an Amazon™ Neptune database snapshot; anAmazon™ Aurora database snapshot; or a Microsoft™ Azure SQL databasesnapshot.
 4. The method of claim 2, further comprising: converting, bythe cloud-based service, the backup data from one or more physicalbackup formats, associated with the plurality of data sources, intological backup data.
 5. The method of claim 4, wherein the storing thebackup data and the metadata in the time-series data lake includes:embedding the metadata into the logical backup data; and storing thelogical backup data, with the embedded metadata, in the time-series datalake in a particular format.
 6. The method of claim 5, wherein theparticular format is one of Apache™ Parquet format, Apache™ Avro format,or Apache™ ORC format.
 7. The method of claim 4, wherein the convertingthe backup data from the one or more physical backup formats intological backup data includes: converting the first physical backup intoa first logical backup of the first data source, wherein the firstlogical backup includes a first plurality of data records from the firstdata source at the first point in time; and converting the secondphysical backup into a second logical backup of the second data source,wherein the second logical backup includes a second plurality of datarecords from the second data source at the second point in time.
 8. Themethod of claim 7, wherein, for a given one of the first plurality ofdata records, the metadata includes a timestamp indicative of the firstpoint in time.
 9. The method of claim 7, wherein, for a given one of thefirst plurality of data records, the metadata specifies the first datasource and the first point in time; and wherein, for a given one of thesecond plurality of data records, the metadata specifies the second datasource and the second point in time.
 10. The method of claim 1, whereinthe backup data includes unstructured data.
 11. A non-transitory,computer-readable medium having program instructions stored thereon thatare executable by one or more computer systems to perform operationscomprising: providing a cloud-based data lake service that maintainsdata for a plurality of organizations, wherein, for a first one of theplurality of organizations, the cloud-based data lake service maintainsa time-series data lake that stores a time-series representation of dataassociated with the first organization; receiving backup data,associated with the first organization, from a plurality of datasources; generating metadata associated with the backup data, wherein,for a given data element of the backup data, corresponding metadataincludes source information identifying a particular one of theplurality of data sources from which the given data element of thebackup data originated; and storing the backup data and the metadata inthe time-series data lake.
 12. The non-transitory, computer-readablemedium of claim 11, wherein the backup data includes: a first physicalbackup, created at a first point in time, of a first one of theplurality of data sources, wherein the first physical backup is providedin a first format; and a second physical backup, created at a secondpoint in time, of a second one of the plurality of data sources, whereinthe second physical backup is provided in a second format.
 13. Thenon-transitory, computer-readable medium of claim 12, wherein theoperations further comprise: converting the backup data from one or morephysical backup formats, associated with the plurality of data sources,into logical backup data.
 14. The non-transitory, computer-readablemedium of claim 13, wherein the storing the backup data and the metadatain the time-series data lake includes: embedding the metadata into thelogical backup data; and storing the logical backup data, with theembedded metadata, in the time-series data lake in a particular format.15. The non-transitory, computer-readable medium of claim 13, whereinthe converting the backup data from the one or more physical backupformats into logical backup data includes: converting the first physicalbackup into a first logical backup of the first data source, wherein thefirst logical backup includes a first plurality of data records from thefirst data source at the first point in time; and converting the secondphysical backup into a second logical backup of the second data source,wherein the second logical backup includes a second plurality of datarecords from the second data source at the second point in time.
 16. Thenon-transitory, computer-readable medium of claim 15, wherein, for agiven one of the first plurality of data records, the metadata includesa timestamp indicative of the first point in time.
 17. A method,comprising: receiving, by a cloud-based service, backup data for anorganization as part of one or more backup operations, wherein thebackup data includes a first backup image, of a first data source, thatwas generated at a first point in time; creating, by the cloud-basedservice, a logical backup of the first data source using the firstbackup image, wherein the logical backup includes a plurality of datarecords from the first data source at the first point in time;generating, by the cloud-based service, metadata associated with thefirst backup image; and storing, by the cloud-based service, the logicalbackup and the metadata in a time-series data lake associated with theorganization.
 18. The method of claim 17, wherein the creating thelogical backup of the first data source includes extracting theplurality of data records from the first backup image such that logicalbackup data in the logical backup is independent of a physical backupformat associated with the first data source; and wherein the methodfurther comprises embedding the metadata into the logical backup data.19. The method of claim 18, wherein the storing the logical backup andthe metadata in the time-series data lake includes: storing the logicalbackup data, with the embedded metadata, in the time-series data lake ina column-oriented format.
 20. The method of claim 17, wherein themetadata includes at least one of: a time stamp corresponding to thefirst point in time; an identifier associated with the first datasource; access control information identifying users with access to oneor more of the plurality of data records; and a schema associated withthe first data source.